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Overview 



The principle guiding the New Hope Project — a demonstration program that was implemented in two inner- 
city areas in Milwaukee from 1994 through 1998 — was that anyone who works full time should not be poor. 
New Hope offered low-income people who were willing to work full time several benefits, each of which was 
available for three years: an earnings supplement to raise their income above the poverty level; subsidized 
health insurance; subsidized child care; and, for people who had difficulty finding full-time work, referral to a 
wage-paying community service job. The program was designed to increase employment and income as well 
as use of health insurance and licensed child care, and it was hoped that children would be the ultimate benefi- 
ciaries of these changes. 

A team of researchers at MDRC and the University of Texas at Austin is examining New Hope’s effects in a 
large-scale random assignment study. This interim report from the study focuses on the families and children 
of the 745 sample members who had at least one child between the ages of 1 and 10 when they entered the 
study. The new findings draw on administrative records and survey data covering the period up to five years 
after study entry (Y ear 5), that is, two years after the program ended. A final report will examine New Hope’s 
effects after eight years. 

Key Findings 

• Employment and Income. Parents in the New Hope group worked more and earned more than did par- 
ents in the control group. Although the effects diminished after Year 3, when the program ended, they did 
persist for some parents. The provision of community service jobs was important to increasing employ- 
ment: 30 percent of program group members worked in a community service job while in New Hope. The 
program reduced poverty rates through Year 5. 

• Parents’ Well-Being. Although New Hope had few effects on levels of material and financial hardship, it 
did increase parents’ instrumental and coping skills. Program group members were more aware of “help- 
ing” resources in the community, such as where to find assistance with energy costs or housing problems, 
and more of them knew about the Earned Income Tax Credit (EITC). They also reported better physical 
health and fewer signs of depression than did control group members. 

• Parenting and Children’s Activities. Although New Hope had few effects on parenting, it did increase 
children’s time in formal center-based child care and after-school programs. Even in Year 5, after eligibil- 
ity for New Hope’s child care subsidies had ended, children in New Hope families spent more time than 
their control group counterparts in center-based child care and after-school programs and correspondingly 
less time in home-based and unsupervised care. New Hope also increased adolescents’ participation in 
structured out-of-school activities, such as youth groups and clubs. 

• Children’s Outcomes. At the end of both Year 2 and Year 5, children in the New Hope group performed 
better than control group children on several measures of academic achievement, and their parents re- 
ported that the children got higher grades in reading and literacy skills. New Hope also improved chil- 
dren’s positive social behavior. All these effects were more pronounced for boys than for girls. 

The New Hope findings support the wisdom of recent expansions in work supports for poor families, including 
increases in the value of the EITC and greater eligibility for Medicaid and child care subsidies. The program’s 
lasting effects on children also have special relevance to the redesign of the nation’s income support system. 
Language proposed in the 2003 reauthorization of the 1996 federal welfare reform legislation would establish 
improving the well-being of poor children as the law’s overarching purpose. The present findings show that 
fulfilling this purpose need not be at odds with the goal of moving parents to work. 
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Preface 



New Hope was an ambitious program based on two simple yet widely held principles: 
People who are willing to work full time should be able to do so, and they should not be poor 
when they do. The program was designed to improve the lives of low-income families by 
providing several benefits for parents who worked full time: an earnings supplement to raise 
their income above poverty, subsidized health insurance, and subsidized child care. The pro- 
gram also offered access to wage-paying community service jobs for people who could not 
find full-time work. 

New Hope was run as a demonstration project fi'om 1994 to 1998 in two inner-city ar- 
eas in Milwaukee, Wisconsin, by the New Hope Project, Inc., a local community-based organi- 
zation. The program had only four eligibility requirements: that applicants live in one of the two 
targeted service areas, be age 18 or older, be willing and able to work at least 30 hours per 
week, and have a household income at or below 150 percent of the federally defined poverty 
level. Participation was voluntary, and adults were eligible regardless of whether they had chil- 
dren and whether they were receiving public assistance. 

New Hope was designed to be replicable as government policy, and one goal of the pro- 
ject was to provide credible information to policjmiakers on the effectiveness and costs of this ap- 
proach. This report is one of a series examining New Hope’s effects on families and children. An 
earlier report presented effects two years after program entry. This report presents findings after 
five years, or two years after program services ended, and it focuses on those adults who had chil- 
dren, the majority of whom were single mothers receiving public assistance when they entered the 
study. Applicants were assigned using a lottery-like process to New Hope or a control group, and 
the program’s effects were estimated by comparing how the two groups fared over time. 

The New Hope evaluation goes beyond economic effects to examine the program’s ef- 
fects on family functioning and children’s well-being. As such, it is an ambitious evaluation, 
using diverse research methods and data and involving a collaboration among MDRC staff. 
New Hope’s board and staff, and prominent university-based scholars. This collaboration was 
begun under the auspices of the MacArthur Foundation Research Network on Successful Path- 
ways Through Middle Childhood. 

The five-year story is one of lasting effects on children’s environments and their well- 
being. At both the two-year and the five-year points, parents in New Hope were more likely 
than control group parents to use center-based child care. It is striking that these effects persisted 
through Year 5, or two years after New Hope child care subsidies had ended. New Hope also 
improved children’s positive behavior and school performance — effects that occurred while 
the program operated and after it had ended. After five years, for example. New Hope children 
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scored higher than children in the control group on a standardized reading achievement test, and 
they received higher ratings from their parents on their performance in school. The effects on 
school performance were larger for boys than for girls, and the effects on positive behavior oc- 
curred primarily for boys. 

What brought about these positive effects on children? The answer probably varies 
across families. First, New Hope increased work and income: Parents in the New Hope group 
worked more and had higher incomes than parents in the control group. Although the effects 
faded for the sample as a whole, they did persist through Year 5 for some families. New Hope 
also reduced poverty through the fifth year. Second, New Hope parents reported lower levels of 
depressive symptoms than control group parents after five years, and they were more aware of 
community resources, including the Earned Income Tax Credit (EITC). It is likely that some 
children benefited from their time in center-based child care and after-school programs, that 
others benefited from the fact that their families had higher incomes because of the earnings 
supplement, and that still other children benefited from some of the other positive changes in 
the home environment. Finally, it is possible, and even likely, that the positive effects on chil- 
dren at Year 5 are a result of the earlier gains, observed at Year 2, which may have set children 
on a permanently higher trajectory. 

New Hope operated during a time when both work supports (in the form of the EITC 
and child care subsidies) and work mandates (in the form of welfare reform) were expanding. 
Within this changing context, the program still increased work and income and made families 
and children better off. The findings show the importance of work supports for low-income 
families and their children and suggest that recent policies that have increased these types of 
supports have been changes in the right direction. 



Gordon Berlin 
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Summary Report 



In today’s labor market, many people who work do not earn enough to lift their families 
out of poverty. Many low-wage workers are not offered health insurance through their jobs, and 
many families with children face prohibitive child care costs. Policymakers have responded by 
implementing and expanding a series of programs designed to “make work pay.” The Earned 
Income Tax Credit (EITC) — a wage supplement that is paid via the tax system to parents with 
low earnings — is one example of a policy designed to support low-income working families. 

The New Hope Project is another example. Conceived by a nonprofit community-based 
organization. New Hope was an innovative program designed to improve the lives of low- 
income people who were willing to work full time, by providing several benefits: an earnings 
supplement to raise their income above poverty, subsidized health insurance, and subsidized 
child care. For people who had difficulty finding full-time work, the program offered help in 
obtaining a job, including referral to a wage-paying community service job when necessary. 
Thus, the project conditioned its benefits on full-time work, because one of its key goals was to 
increase employment, but it offered supports to help people meet this requirement. In the dem- 
onstration project, each of the benefits was available for three years.' 

New Hope was run in two inner-city areas in Milwaukee, Wisconsin, and had only four 
eligibility requirements: that applicants live in one of the two targeted service areas, be age 18 
or over, be willing and able to work at least 30 hours per week, and have a household income at 
or below 150 percent of the federally defined poverty level. Participation was voluntary, and 
adults were eligible regardless of whether they had children and whether they were receiving 
welfare. A team of researchers at MDRC and the University of Texas at Austin is evaluating 
New Hope’s effects. In the evaluation, applicants to the program were assigned through a lot- 
tery-like process either to New Hope or to a control group, and the program’s effects were esti- 
mated by comparing how the two groups fared over time. This report focuses on those adults 
who had children, the majority of whom were single mothers receiving public assistance when 
they entered the study. 

The idea for New Hope originated in the early 1 990s in response to stmctural factors in 
both the labor market and the welfare system that worked against low-income families. Welfare 
mles — which reduced benefits nearly one dollar for every dollar increase in earnings — along 
with low wages in the labor market and the absence of employer-provided medical insurance 
meant that leaving welfare for work did not always make families better off financially and of- 
ten led to the loss of health coverage. After paying for child care and other work expenses, low- 



' Parents who needed community service jobs could work in these jobs for up to 12 months. 
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income families were sometimes worse off, and they were usually still living below the poverty 
level. New Hope’s underlying principles are that people who are willing to work full time 
should be able to do so and that they should not be poor when they do. 

Each of New Hope’s components is similar in some respects to features of other pro- 
grams that now help poor families, such as child care assistance, subsidized health coverage, 
and the EITC. In fact, the program, which ran from 1994 through 1998, operated during a time 
in which the policy environment faced by low-income families was changing dramatically, in 
terms of work mandates, work supports, and work opportunities. The EITC, for example, was 
increased substantially during the 1990s, making it now one of the largest antipoverty programs 
in the country. Both Medicaid and child care assistance were expanded, and eligibility was ex- 
tended to low-income families not receiving welfare. The welfare system was also beg innin g a 
major period of reform, which culminated with the 1996 legislation requiring work mandates 
and time limits. The State of Wisconsin was in the forefront in the effort to reform welfare and 
had imposed work requirements for recipients even prior to the national legislation. The state 
has also been in the forefront on the work supports side, providing a generous state EITC and 
significantly expanding funds for child care assistance and health coverage. Finally, these in- 
creases in work mandates and work supports occurred during a period of strong economic 
growth, when unemployment rates both nationally and in Milwaukee were low. 

Within this changing context. New Hope was unique in that it offered one package — 
administered conveniently in one setting — containing a suite of benefits and services that par- 
ents could take up and use according to family needs and preferences. Importantly, it also had 
higher income thresholds than then-existing health, child care, and earnings subsidy programs. 

New Hope was designed to be replicable as government policy, and one goal of the pro- 
ject was to provide credible information to policymakers on the effectiveness and costs of this 
approach. New Hope’s designers expected that its combination of benefits and services would 
have the direct effects of increasing parents’ employment and their use of health insurance and 
licensed child care. These effects, in turn, might influence the well-being of the program’s 
adults and their families. It was hoped that the ultimate beneficiaries of the program would be 
the children, whose development might benefit from reduced poverty, changes in the home en- 
vironment, and increased time in licensed child care. Thus, the evaluation set out to answer sev- 
eral questions: How many eligible families would make use of New Hope’s benefits? Would 
the program increase employment and reduce poverty? Would it affect other aspects of parents’ 
well-being and the lives of their children? Finally, would New Hope affect children’s develop- 
ment and well-being? 
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The Evaluation 



This report is part of a series on the New Hope Project. Early reports examined the im- 
plementation of the program and participants’ use of services. The most recently published re- 
port examined the effects on parents and children two years after parents applied for the pro- 
gram, when the study children were 3 to 12 years old.^ The current report presents results five 
years after application to the program, when the children were 6 to 1 6 years old. A subsequent 
report will examine the program’s effects after eight years. 

New Hope was evaluated using a random assignment research design. After conducting 
outreach in the communities to identify eligible people, the study enrolled over 1,300 low-income 
adults. Half the applicants were randomly assigned to a program group that was eligible to receive 
New Hope’s benefits, and the other half were randomly assigned to a control group that was not 
eligible for the enhanced benefits. Because the random assignment process created two groups 
that closely resemble each other, comparing outcomes such as employment and income for the 
two groups over time gives a reliable estimate of New Hope’s effects. From the total sample of 
1,357 people, 745 people had at least one child between the ages of 1 and 10 at the time of en- 
rollment. These families constitute the Child and Family Study (CFS) sample and are the focus of 
this report.^ Almost 90 percent of the adults in this sample were single or separated mothers with 
children when they entered the study, and 80 percent were receiving public assistance. 

The evaluation makes use of several data sources. New Hope program data provide in- 
formation on parents’ use of the program’s services. State administrative records provide data 
on employment and receipt of welfare and food stamp benefits. In-person surveys — adminis- 
tered to parents and children at two years and five years after random assignment — obtained 
information on families’ receipt of New Hope benefits, parents’ employment and earnings, fam- 
ily functioning, and parent-child relations. For up to two “focal” children in each family, the 
surveys also collected information from both parents and children on participation in child care 
and other activities as well as children’s behavior and school performance.'* Teachers were 
mailed surveys asking about the children’s school performance and social behavior. 

In order to understand in depth the dynamics of family life, the experiences of parents 
and children, and the contexts in which families lived and worked, an ethnographic sample of 
44 families was drawn from the CFS sample. These families — half of whom were in the New 
Hope group and half of whom were in the control group — were followed from the third year. 



^J. M. Bos, A. C. Huston, R. Granger, G. Duncan, T. Brock, and V. McLoyd, New Hope for People with Low 
Incomes: Two-Year Results of a Program to Reduce Poverty and Reform Welfare (New York: MDRC, 1999). 
separate report on the total sample is forthcoming. 

‘'if there were more than two children in the family between the ages of 1 and 10, two were chosen at ran- 
dom to be the focal children. 



or the final year of the New Hope program, through the fifth year. The ethnographic data in- 
clude extensive field notes as well as focused interviews covering a wide range of topics, in- 
cluding, for example, parents’ experiences with New Hope, family routines, work experiences, 
family relationships, child care arrangements, and goals. 



Findings 

The findings show that work supports can have a range of positive effects on low- 
income families and their children. First, New Hope increased work and income: Parents in the 
New Hope group worked more and had higher incomes than parents in the control group. Al- 
though these effects on work and income faded for the sample as a whole after Year 3, when the 
program ended, to a remarkable extent New Hope continued to have positive effects on a wide 
range of other outcomes throughout the five-year follow-up. For example. New Hope partici- 
pants had more stable employment, lower rates of poverty, and higher wages at the five-year 
point. New Hope parents also reported lower levels of depressive symptoms than control group 
parents after five years, and they were more aware of community resources, including the EITC. 
In addition to affecting parents’ economic status. New Hope influenced children’s environments 
by increasing their time in center-based child care and in other structured activities. New Hope 
parents were more likely than control group parents to use center-based child care — an effect 
that persisted through the fifth year, or two years after New Hope child care subsidies had 
ended. Finally, both while the program operated and at five years after study entry. New Hope 
improved children’s positive social behavior and performance in school. New Hope children 
scored higher than children in the control group on a standardized reading achievement test, and 
they received higher ratings fi-om their parents on their performance in reading and literacy at 
school. The effects on teachers’ reports of school performance and positive behavior occurred 
primarily for boys. 

Because New Hope’s benefits were offered as a package, it is not possible to determine 
whether one particular component led to the program’s positive effects on families and children. 
In fact, the findings suggest that New Hope had the effects it did because it offered a menu of 
benefits, allowing families to choose and use services according to their needs. It is likely that 
some children benefited fi-om their time in center-based child care and after-school programs; oth- 
ers benefited fi-om the fact that their families had higher incomes because of the earnings supple- 
ment; and still others needed the extra support of community service jobs to get them started. 

The Use of New Hope’s Benefits 

• The majority of families in the program group used New Hope ser- 
vices at some point during the three-year eligibility period, although 
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few families received benefits every month. The annual cost of pro- 
viding these benefits was $5,300 per family. 

The large majority (87 percent) of people in the program group received at least one 
New Hope benefit (earnings supplement, health insurance, or child care assistance) during the 
three-year period. However, in any given month, between 40 percent and 50 percent of the 
sample were receiving at least one type of benefit — the most common being the earnings sup- 
plement. In addition, families who did receive benefits received them for an average of 17 
months out of the 36-month eligibility period. The fact that most families did not use benefits 
consistently is partly the result of New Hope’s design; for example, benefits were offered only 
to full-time workers, and the value of benefits decreased as family income increased. In addi- 
tion, some families did not need particular benefits, such as low-cost health coverage, if they 
received Medicaid or employer-provided coverage. The use of benefits was also related to fami- 
lies’ circumstances. According to the ethnographic data, families were most likely to use bene- 
fits when they could strategically manage work and finances and when benefits complemented 
their current arrangements. Families who were struggling with health and personal problems 
were less likely to work full time and less likely to use benefits. The average aimual cost of pro- 
viding these benefits was $5,300 per program group family, and the average family consisted of 
one adult and two or three children. The largest component of these costs was New Hope’s 
child care subsidies. Half the families who were still receiving benefits when the program ended 
reported being able to adjust to the loss of these benefits — in part owing to the efforts of New 
Hope staff, who worked to inform families of the upcoming end of benefits and to ensure that 
this adjustment went as smoothly as possible. However, about 20 percent of these families re- 
ported major difficulties adjusting to the loss of benefits. 

• New Hope staff services provided positive support for participants. 

Two years after random assignment, when families were still eligible for New Hope 
benefits, parents in the program group reported receiving higher levels of instrumental and emo- 
tional support — probably a reflection of New Hope staff services. In ethnographic and focus 
group interviews, parents praised the staff for respectful and helpful assistance. 

• By the fifth year, or two years after the end of eligibility for New 
Hope benefits, the program group and the control group received 
similar levels of benefits from public sources. 

Two years after they entered the program, when benefits were still available, more 
adults in the program group than in the control group were covered by health insurance and 
were receiving child care subsidies — a direct result of the program’s benefits. By the five-year 
point, however, or two years after New Hope ended, there were no differences between the two 
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groups on these outcomes. Some families in both groups were making use of the expanded eli- 
gibility for these types of work supports, particularly with respect to health coverage. 

Effects on Employment and Income 

• New Hope increased employment and earnings. 

Parents in the New Hope group worked more and earned more than did those in the 
control group (Figure 1). For the sample as a whole, the effects occurred largely during the first 
three years, when the program was still operating. For some groups, however — such as His- 
panic parents and parents who faced moderate barriers to employment (for example, large fami- 
lies, preschool-aged children) — the impacts on employment and earnings lasted throughout the 
five-year period. The impacts on employment and earnings, especially in the early years, would 
not have been as large if New Hope had not provided community service jobs to those who 
needed them: 30 percent of parents in the program group worked in a community service job at 
some point during the first three years. 

• New Hope increased stable employment and average wages. 

Parents in the New Hope program worked more consistently than did those in the con- 
trol group. This increase in stable employment may have contributed to one of the program’s 
more lasting effects: During the fifth year, the program group earned higher wages than the con- 
trol group. For example, 26.6 percent of people in the program group earned more than $11 per 
hour, compared with 20 percent of people in the control group. 

• New Hope had no effect on welfare receipt. 

Welfare receipt fell dramatically during the five-year period for all groups in the evalua- 
tion sample, reflecting trends nationwide and in Wisconsin. However, rates of receipt were 
similar for the New Hope and control groups. 

• New Hope increased income and reduced poverty. 

The families in the New Hope group had higher incomes than the families in the control 
group, although the effects occurred mostly during the first three years. The program did con- 
tinue to have large effects on income in Years 4 and 5 for some groups in the sample, such as 
Hispanic parents and parents who had only moderate barriers to employment. In contrast. New 
Hope’s effects on poverty persisted throughout the five-year period for the entire sample, due in 
part to the fact that the effects on income — although not statistically significant in the later 
years — were still positive, and in part to the fact that New Hope reduced the number of fami- 
lies who had very low incomes. In Year 5, for example, 52 percent of the program group had 
incomes below the poverty line, compared with 60 percent of the control group (Figure 2). 
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The New Hope Project 
Figure 1 

Impacts on Employment 

New Hope increased employment, but the effects diminished after program services ended 
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Quarter After Random Assignment 

SOURCES: MDRC calculations using data from the New Hope Project MIS client-tracking database and 
Wisconsin unemployment insurance (UI) records. 



NOTE: Only for Quarters 1 through 8 and 13 are the differences between the program and control group 
outcomes statistically significant: Quarters 1 through 4, at the 1 percent level; Quarters 5 through 7 and 13, at 
the 5 percent level; and Quarter 8, at the 1 0 percent level. 
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The New Hope Project 
Figure 2 

Impacts on Poverty 

New Hope reduced the number of families below poverty 
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Year After Random Assignment 




SOURCES: MDRC calculations using data from the New Hope Project MIS client-tracking database and 
Wisconsin unemployment insurance (UI) records. 

NOTES: The differences between the program and control group outcomes are statistically significant for all 
years: Years 1 and 3, at the 1 percent level; Years 2 and 4, at the 5 percent level; and Year 5, at the 10 percent 
level. 

The poverty rates shown here are based on an income measure that includes earnings, EITC benefits, New 
Hope supplements, and public assistance as captured by administrative records. Because these rates do not 
include other sources of household income, they are not comparable to the official poverty rate. 
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Effects on Families’ Well-Being 

• New Hope had few effects on material well-being. 

Although New Hope did reduce the number of families in poverty, at both the two-year 
and the five-year points, the program and control groups reported similar levels of material hard- 
ship, such as food insecurity and financial worries. They also provided similar ratings of the qual- 
ity of their housing and neighborhoods. (Table 1 presents selected effects on parents’ well-being.) 

• Yet the program did have some positive effects on other aspects of 
parents’ well-being and on their instrumental and coping skills. 

Parents in the New Hope group were more aware of available “helping” resources in 
the community, such as where to find assistance with energy costs or housing problems. More 
of them also knew about the EITC, an important source of support for low-income workers. 
Ethnographic data suggest that a significant number of families intentionally used the EITC as a 
savings plan for making major purchases, reducing debt, and stabilizing rent and other pay- 
ments. Parents in New Hope also reported better physical health and fewer symptoms associ- 
ated with depression than did parents in the control group. At the two-year point. New Hope 
parents reported reduced stress, increased feelings of social support, and increased time pres- 
sure. The ethnographic study found that many parents had children with disabilities or behav- 
ioral difficulties; New Hope helped the parents achieve a difficult balance among work, ser- 
vices, and parenting. 

Effects on Children’s Environments 

• New Hope had few effects on parenting and parent-child relations. 

Overall, at the two-year and the five-year points, there were few differences between 
the program and control groups on several measures of parenting, such as parenting stress, pa- 
rental warmth, and regularity of family routines. The New Hope parents did report fewer prob- 
lems controlling their children, and parents of adolescents reported more effective management 
(better control and less need for punishment). 

• New Hope substantially increased children’s time in formal center- 
based child care and in after-school programs. 

Even during the fifth year, after eligibility for New Hope’s child care subsidies had 
ended, children in New Hope families spent more time in center-based child care and after- 
school programs than did children in control group families and correspondingly less time in 
home-based and unsupervised care, which includes self-care and care by siblings. As expected, 
these effects occurred only for children age 12 or younger (Figure 3). Ethnographic data indi- 




Sum-9 



25 



o 

u 

cu 

a. 

o 

X 

z, 



H 



G 

'S 



u 

ea 

cu 

c 

o 



es 

o. 

E 



ig 

•h 



0^ 

IS 

wa 

3 

2i 

e 

*i^ 

:<3 

=5 

-«M 

S 

•c 

bo 

e 

•c 






i 

bt 

c;, 

s: 



b) 

<Si, 

■bt 

2 

I 

I 



O V 



V 

*- w 

O c 

CO <L> 

o. 

E 



c S 

o o 

U 



E §J 
2 o 

p a 



o 



o 



VO 

VO 



o 

o 



00 

d 



00 

d 



o 

d 



<N 

O 



oo 

Ov 



bx 

c 



6> a. ' 



^ 1 

.5 « 

5 1 

CO CO 

s s 



ox 
bx .E 
.S 3 



w .ti 



VO 

d 



oo 

m 

o 



00 

d 



»n 

rn 

rn 



VO 

O 

d 



VO 



o 

d 



oo 

Ov 



<N 

(N 



CN 

O 



OV 

o 



00 

00 



oo 

Ov 









JS 


JS 






c/3 


ox 


ox 


O 


V 

E 


O 

Jl 


IE 

It 


2 

II 

VO 


o 

ox 

c 

cd 


o 

u 

3 


d 

c 

II 


_o 

T 


T 



o 



2 II 

O - (N 

t: 

o « o 

CO > .is 



Cl 

O 



O 

cd 

pfi 

V 

JO 



e 

Cd 

CA 

0> 

*bx 

4> . 



<N 

o 



l/v — 

VO Ov 

00 o 



— to 



p 

d 



oo 



Ov 



VO 

00 



o 

E 

o' o 

E ■£ 

•z: c<.H -c 
o o .SP 
x: — 

c: « A 

O to VO 

u o .> 

c c > 

§11 

Im 



ox o 

.£ *9 

D. ss 



-rr -E >v 



g 



Cd 



C tS 

M 

T3 ^ 

.Z c/3 
W5 • — 

/S « 

K C/) 



— 

5 

M .E 



:§ ° .2 



Xi 

cd 

c 



J= 

eu 



jz 

a. 



o 

U 



< 



O c/3 
c/3 « 
O Cd 

*S W 
^ S 
^ o 



c 

o 



o. 

u 

Q 



a. 

o 

!X 

o 



ox 

c 



U 

Q 

S 

ui 

u 

eci 

D 

O 

c/2 



o 

E 

o 



c 

o 



> 

V 

T3 



T3 

C 



c 

o 

2 

o 

o. 

o 



T3 

c 

cd 



E 

TD 

C 

cd 



OX 

c 



ox 

ox 



5^ 

o 

> 



0) ^ 
a, 60 
c 



VO 



§ iJ 

■o: a, 

° I 
§• “ 
O. O 
Cd |5 

w <u> 
cd 

” <L> 

c/3 ^ 

O 

E 
o 
o 



c/3 



C 

o 

2 =3 

fli v'J 

a. e 



" Cd 

* >> 

* C 

♦ cd 

(/) ^ 

"O cd 

« E 

5 s 

"i 3 

C c/3 



3 

"O 



T3 

C 



.Is 

J2 

Cd o 

« 'E 

z: cd 
cd v/3 

w 73 

2 

ori o 
tu < 
H 
O 
2 



_ E 

o .£ 

D. - ° 

3 P W 

O ^ C 

Ui O ‘O 

OX ih 3 

E "O 2 

c .S ^ 

O Cd 

^ E 

T3 O 

S c« 

E I 

Cd ^ 

OX ™ 

O c^ 

O. c 

C .2 

8 2 

^ > 

o 2 
^ -O 

O "H 

2 ^ 

O f- 

^ 3 

c2 ” 

C2 

^ .i2 
« £ 



T3 

fc' 

<U 

> 

o 



u 

ox 

cd 

C 

cd 

E 

o 

'*S 1 

O 2 

x: c" 

o 

O T3 



u 



> 

o 

o. 

•o 



N 

- *1 

8 P 



Cd 



c/3 



;C O 

b). 

O 

JO 



cd 

O 

OX 



c 

o 



ji: 

o 

cd 



er|c 



Sum- 10 



26 



The New Hope Project 
Figure 3 

Impacts on Child Care 

New Hope increased children's time in formal care through Year 5, or two years after 
the program ended; as expected, impacts occurred only for children age 12 or younger 




SOURCE; MDRC calculations using data from the New Hope five-year survey. 

NOTE; Only for the younger two age groups are the differences between the program and control group 
outcomes statistically significant; children ages 6 to 8, at the 5 percent level; and children ages 9 to 12, at the 
10 percent level. 



cate that the stability of child care arrangements was higher for New Hope families than for 
control group families for about a year after benefits ended. Center-based care and stable center 
care may have contributed to parents’ stability of employment and to children’s academic and 
social skills. 



• New Hope increased children’s participation in some structured 
activities. . 

Two years after New Hope began, children in program group families participated in 
more structured out-of-school activities, such as team sports and youth groups or clubs. When 
these same children were adolescents, they still spent more time than control group adolescents 
participating in such structured out-of-school activities (Table 2). Program group children of all 
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The New Hope Project 
Table 2 

Impacts on Children's Activities 



New Hope increased adolescents* participation in structured activities outside school 



Outcome 


Program 

Group 


Control 

Group 


Impact 

(Difference) 


P-Value 


Effect 

Size“ 


Structured out-of-school activities 












during the school year^ 












All children 


2.42 


2.33 


0.10 


.218 


0.10 


By age at survey 












6 to 8 


2.16 


2.17 


0.00 


.973 


0.00 


9 to 12 


2.60 


2.47 


0.13 


.261 


0.13 


13 to 16 


2.57 


2.27 


0.30 ♦♦ 


.029 


0.32 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and ♦ = 10 percent. 

For all children, parent reports were available for 830 children. Actual sample sizes for individual measures 
may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the 
standard deviation of the outcomes for both groups combined. This standard deviation is always obtained from 
the entire research sample, even for subgroups. 

Using a scale that ranged from 1 ("never") to 5 ("about every day"), parents reported on children's 
participation in such activities as organized sports, religious classes and events, clubs, and lessons. 



ages participated in activities offered by religious organizations more than did control group 
children (not shown in table). 

Effects on Children 

• New Hope improved children’s school performance. 

At both the two-year and the five-year points, children in the New Hope group performed 
better than control group children on several measures of academic achievement, particularly on 
reading and literacy tests (Table 3). After five years, they scored higher on a standardized test of 
reading skills, and their parents reported that they got higher grades in reading skills. These effects 
were slightly more pronounced for boys than for girls. Compared with their control group coun- 
terparts, boys in New Hope also received higher ratings of academic performance from their 



The New Hope Project 
Table 3 

Impacts on Children's Academic Achievement 



New Hope improved children 's school performance 



Outcome 


Program 

Group 


Control 

Group 


Impact 

(Difference) 


P-Value 


Effect 

Size“ 


Woodcock-Johnson test of reading achievement^ 


All children 


98.05 


96.01 


2.05 * 


.091 


0.12 


Boys 


97.74 


94.85 


2.88 


.106 


0.18 


Girls 


98.71 


96.94 


1.78 


.263 


0.11 


Parents' ratings of reading achievement^ 


All children 


3.70 


3.48 


0.22 *** 


.006 


0.19 


Boys 


3.52 


3.29 


0.23 ** 


.047 


0.20 


Girls 


3.88 


3.69 


0.19 * 


.099 


0.16 


Teachers' ratings of academic achievement^ 


All children 


3.21 


3.15 


0.06 


.517 


0.06 


Boys 


3.22 


2.92 


0.30 ** 


.035 


0.30 


Girls 


3.19 


3.36 


-0.17 


.171 


-0.17 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Test results were available for 816 children, parent reports were available for 830 children; and teacher 
reports were available for 53 1 children. Actual sample sizes for individual measures may vary as a result of missing 
data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even for subgroups. 

*^Woodcock- Johnson scores are age-standardized with a mean of 100 and a standard deviation of 15. 

‘^Parents' ratings ranged from 1 ("below average") to 5 ("excellent"). 

‘^Teachers' ratings on the academic subscale of the Social Skills Rating System ranged from 1 ("lowest 10 
percent of the class") to 5 ("highest 1 0 percent of the class") and covered children's performance in reading, math, 
intellectual functioning, motivation, oral communication, classroom behavior, and parental encouragement. 
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BEST 



The New Hope Project 
Table 4 

Impacts on Children's Behavior 



New Hope improved children '5 positive behavior, primarily for boys 



Outcome 


Program 

Group 


Control 

Group 


Impact 

(Difference) 


P-Value 


Effect 

Size" 


Positive Behavior Scale’’ 












All children 


Parent reports 


3.89 


3.81 


0.08 * 


.061 


0.15 


Teacher reports 


3,60 


3.59 


0.01 


.915 


0.01 


Boys 


Parent reports 


3.83 


3.76 


0.07 


.207 


0.13 


Teacher reports 


3.58 


3.42 


0.17 * 


.078 


0.24 


Girls 


Parent reports 


3.95 


3.87 


0.08 


.161 


0.15 


Teacher reports 


3.60 


3.77 


-0.18 ** 


.037 


-0.26 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

For all children, parent reports were available for 830 children; and teacher reports were available for 531 
children. Actual sample sizes for individual measures may vary as a result of missing data, 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even for subgroups. 

’'The Positive Behavior Scale of the Social Skills Rating System includes 25 items divided into three 
subscales: compliance and self-control, social competence and sensitivity, and autonomy. Both parents and 
teachers completed these scales (l=”never," 5="all of the time"). 



teachers and were more likely to expect to attend college at both the two-year and the five-year 
assessments. New Hope adolescents reported more engagement vAth school, feelings of efficacy, 
and expectations to finish college than did their control group counterparts. 

• New Hope improved children’s positive social behavior, especially for 
boys. 

At the five-year point, parents in New Hope rated their children higher on positive so- 
cial behavior — including compliance, self-control, and sensitivity — than did parents in the 
control group (Table 4). Boys in New Hope also received higher ratings from their teachers in 
terms of appropriate classroom behavior and positive social behavior, and they responded to 
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hypothetical scenarios of peer provocation with less hostility than did control group boys, sug- 
gesting that they were less aggressive. Girls in New Hope, in contrast, received lower ratings 
from their teachers on classroom and social behavior and higher ratings on problem behaviors 
than did their control group counterparts. There were no effects on children’s reports of delin- 
quent actions or other risky behavior for either boys or girls. 

The New Hope findings support the wisdom of recent expansions in work supports for 
poor families, including increases in the value of the EITC and expansions in eligibility for 
Medicaid and child care subsidies. New Hope benefits were added to a range of work supports 
that already existed in Wisconsin, and it was not clear in this context whether they would have 
any added effect. The results of this study show that tliey did. In addition, the program’s lasting 
effects on children have special relevance to the redesign of the nation’s income support system 
for low-income families. Language proposed in the 2003 reauthorization of the 1996 Tempo- 
rary Assistance for Needy Families (TANF) welfare reform legislation would establish improv- 
ing the well-being of poor children as the overarching purpose of welfare reform. TANF’s block 
grant structure gives states a lot of flexibility in how these fimds are used to support poor fami- 
lies. The New Hope findings suggest one possible means of achieving TANF’s new purpose 
and show that the goals of increasing family income and helping poor children need not be in- 
consistent with the goal of moving parents to work. 



Pathways of Effects on Children 

How and why did New Hope lead to lasting gains for children? Table 5 presents a 
summary of the program’s effects. New Hope was designed to increase parents’ employment 
and families’ material resources, and it did. Although the program’s impacts on employment 
and income faded somewhat for the sample as a whole after Year 3, New Hope participants had 
more stable employment, lower rates of poverty, and higher wages at the five-year point. Stable 
employment and modestly higher income may have increased family resources available for the 
children (for example, center-based child care) and may have improved parents’ psychosocial 
well-being. Program group parents reported better physical health and slightly lower levels of 
depressive symptoms than control group parents, and these measures are both indicators of 
adult well-being. Perhaps more important, there is some evidence that program group parents 
were more aware of community resources and of the EITC. 

Changes in parents’ employment and family income are likely to affect children 
through their impacts on everyday experiences at home and away from home. Although there is 
a great deal of evidence from other research showing that income affects parents’ well-being, 
which in turn contributes to positive parenting, in fact this study found only very modest evi- 
dence of program impacts on parenting practices. 





The New Hope Project 
Table 5 

Summary of New Hope’s Impacts 


Outcome 


Program Group Versus Control Group 


Parents’ employment and income 


Modestly higher income 
Less poverty 
More stable employment 
Higher wages 


Parents’ well-being 


No difference in material or financial well-being 

Slightly better physical health 

Fewer depressive symptoms 

Better awareness of public and community resources 

Better able to sustain daily routine 


Parenting 


Few overall effects 

Fewer problems with control in discipline situations 
Boys: More positive parent relations 
Adolescents: More effective child management 


Child care 


More center-based care 
More after-school programs 
Less home-based care 
Less unsupervised care 
Fewer changes in arrangements 


Children’s out-of-school activities 


More participation in religious activities and organizations 
Adolescents: More participation in structured activities (for example, 
sports, lessons, community centers) 


Children’s academic achievement 


Better scores on standardized reading achievement test 
Better reading performance (as reported by parents) 
Boys: Better academic skills (as reported by teachers) 



Children’s motivation and well-being No overall impacts 





Boys: Higher educational expectations 

Greater school engagement 
Adolescents: 

Higher educational expectations 
Greater school engagement 
Increased feelings of efficacy to reach goals 
Greater future community involvement 


Children’s social behavior 


More positive social behavior (as reported by parents) 

No difference in risky, delinquent behavior 

Boys: More positive social behavior (as reported by teachers) 

More appropriate classroom behavior (as reported by teachers) 
Less hostility in provocation situations 
Girls: Less positive social behavior (as reported by teachers) 

More problem behavior (as reported by teachers) 


Children’s health 


No impacts 



Parenting involves more than direct interaction with children; parents affect their chil- 
dren by the arrangements they make for children’s experiences in school, the community, and 
other settings. New Hope had strong impacts on children’s experiences outside the family over 
the entire five-year period. Even though New Hope child care subsidies ended after three years, 
parents continued to use more formal center-based and after-school child care during the school 
year and more formal care during the summer. By contrast, control group children were more 
likely to be unsupervised and to be cared for by a minor during the summer. New Hope children 
also had more stable child care arrangements than control group children. Previous research has 
concluded that stable, center-based child care and after-school programs contribute to children’s 
academic performance.^ 

It is striking that parents in New Hope continued to use formal child care after their eli- 
gibility for child care benefits had ended and that older children in the program continued to 
participate in more structured activities. One reason may be that program group parents had 
gained sophistication about the public and private resources available to them, which may be an 
important and enduring legacy of the high-quality information and assistance they had received 
fi’om New Hope project representatives. New Hope parents may have been more proactive than 
control group parents in using a range of programs and services, particularly for older children. 

The sustained impacts of New Hope on children’s academic performance may have re- 
sulted fi’om the lasting effects of the program on children’s environments at home and away fi’om 
home. But the long-term gains may also have resulted fi’om advantages accrued during the three- 
year benefit period that led to an upward spiral. The better school performance (as rated by teach- 
ers) that New Hope children demonstrated at the two-year point could have led to experiences of 
success, positive attitudes about school, and positive perceptions by teachers that were self- 
perpetuating. The initial treatment-induced changes in children’s behavior may also have affected 
their home and school experiences, either by eliciting particular reactions fi’om the people around 
them or by leading the children to seek out different activities, settings, and peers. 

The positive effects were more pronounced for boys than for girls. Considering that, on 
average, boys are more “at risk” than girls, particularly in low-income families, these effects could 
be very important. Control group boys had lower levels of academic achievement and positive 
behavior than did control group girls. Ethnographic data suggest that parents were especially con- 
cerned about the dangers facing their boys and that they used the extra resources fi’om New Hope 
to provide material goods and positive experiences for boys. At the same time, there were nega- 
tive impacts on teachers’ ratings of girls — a pattern that increased in magnitude after two years. 
The reasons are not clear, but these findings raise concern about girls’ relations to school. 



^NICHD Early Child Care Research Network (2000), “The Relation of Child Care to Cognitive and Lan- 
guage Development,” Child Development 1\ \ 960-980. 
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These results suggest some of the possible pathways by which New Hope may have af- 
fected children, but they also illustrate the difficulty of pinpointing any one factor. Although time 
in child care and stmctured activities seem to be important factors, the results as a whole suggest 
that there may have been multiple paths of influence. Some parents had consistently higher in- 
comes because of New Hope; others used stable, center-based child care; and still others experi- 
enced improved psychosocial well-being. All these impacts — alone or in combination — could 
have improved children’s well-being. This is consistent with the fact that differences across sub- 
groups in the program’s impacts on earnings and income, for example, did not translate into simi- 
lar differences in the program’s impacts on children. In addition, the ethnographic researchers ob- 
served varying responses to the New Hope offer as well as different life trajectories across the 
sample. It appeared that, by offering a choice of benefits. New Hope enhanced the overall ability 
of some program group families to find greater stability — and thus to sustain their daily routines 
amid the cascade of problems that so often overwhelm working-poor families. 

That there are likely multiple pathways is also suggested fi-om the fact that the New 
Hope offer included a “cafeteria” of supports, which enrollees could assemble into a customized 
package to meet their specific needs. The core benefits included a wage supplement, access to 
community service jobs, and assistance with both child care and health insurance. These con- 
crete supports were “wrapped” in a variety of less tangible supports, such as the relationship 
with a New Hope project representative, who worked with the enrollee in administering the core 
benefits, as well as workshops on practical topics and informal get-togethers with other enrol- 
lees. Enrollees could avail themselves of these “softer” benefits as they saw fit. The diversity of 
the backgrounds and characteristics of the New Hope population, coupled with the project’s 
“cafeteria-style” set of supports, meant that parents were able to use the benefits in ways that fit 
their overall circumstances and preferences. 



Implications for Policy 

• As expected, the employment and income effects of a work support 
program are largest and affect the broadest range of people during 
the period in which the program operates. 

The original vision of New Hope was a program of continuous work supports, but fund- 
ing constraints ultimately made it a test of a three-year “dose” of benefits. For the sample as a 
whole. New Hope’s effects on earnings and income were the largest and most compelling dur- 
ing Years 1 through 3. This pattern of results was also found for a wage supplement program in 
Canada, in which the positive impacts on employment and earnings faded after the supplement 



payments ended.® Impacts on employment and earnings fade over time in most programs, usu- 
ally because employment rates for the control group eventually catch up with rates for the pro- 
gram group. Because of the strong economy during this five-year follow-up period for New 
Hope, people in the control group could find work fairly easily. In this case, the eight-year fol- 
low-up will be important in assessing whether the increased work experience for people in the 
New Hope group (which included a sizable increase in stable work) helps them weather the 
weaker labor market of recent years. 

Would New Hope’s impacts have lasted longer if its benefits had been extended by 
several years or even indefinitely? The program’s effects on income probably would have per- 
sisted, because families could continue receiving the earnings supplement, but its effects on 
employment may or may not have continued. On the one hand, the pattern of employment im- 
pacts suggests that people who went to work because of the New Hope offer did so fairly 
quickly. On the other hand, extending the supplement beyond three years might have encour- 
aged some New Hope parents who left work after the three-year mark to find new jobs or to 
find them more quickly. Also, community service jobs were important in generating the early 
employment effects. Offering this component after three years might have increased employ- 
ment among parents who could not find full-time work. 

• Nonetheless, providing even a temporary package of work supports 
to low-income parents can have long-term positive effects for chil- 
dren. 

New Hope led to positive effects on children’s school performance and behavior at the 
two-year and five-year points. The impacts on children in Year 5 are especially persuasive be- 
cause they appeared on measures obtained from multiple sources — parents, teachers, chil- 
dren’s reports, and standardized tests. 

But are the impacts large, and will they really affect children’s longer-term well-being? 
Although New Hope’s effects on school achievement were not large in an absolute sense, they 
are sufficiently large to be socially important, given the long time period between the program 
and the measured outcomes. The average child in New Hope scored above 54 percent of the 
children in the control group on a standardized reading test. The long-term gain in reading and 
math achievement produced by New Hope was about one-third the size of the gain produced by 
the Abecedarian program (a much-acclaimed, very intensive and expensive five-year early in- 
tervention program implemented in the 1970s). Differences of this magnitude may lead to in- 
creased probability of completing high school and post-high school education. The fact that the 

®C. Michalopoulos, D. Tattrie, C. Miller, P. K. Robins, P. Morris, D. Gyarmati, C. Redcross, K. Foley, and 
R. Ford, Making Work Pay: Final Report on the Self-Sufficiency Project for Long-Term Welfare Recipients 
(Ottawa: Social Research and Demonstration Corporation, 2002). 
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impacts did not disappear after the two-year mark also suggests that they represent a shift to 
new trajectories that could continue in future years. 

New Hope’s effects are consistent with findings fi"om other programs that improved 
children’s outcomes and also increased parents’ earnings and incomes by providing wage sup- 
plements.’! n addition, the New Hope findings suggest that a key part of this strategy may be 
subsidized child care. New Hope created large effects on the use of formal center-based child 
care and out-of-school programs, which may have contributed to the improvements in academic 
success and positive social behavior for participants’ children. These results suggest that both 
children and adolescents would benefit if these types of child care and activities were more 
readily available to low-income families. 



*** 

New Hope was offered during a time in which both work supports and work mandates 
were expanding rapidly for low-income families, particularly in Wisconsin. When placed in this 
context, the program still encouraged more parents to go to work — increasing their earnings 
and incomes — and it enhanced the well-being of their children. The effects are all the more 
impressive, given the strong economy during the period and given the fact that New Hope was 
an entirely voluntary program. The findings support the wisdom of recent policies that have in- 
creased the value of the EITC and begun to extend eligibility for Medicaid and child care subsi- 
dies. Unfortunately, this trend may be reversed in the next several years, given the budgetary 
pressures faced by states in the early part of the decade. 

The New Hope findings also suggest that the goal of helping poor families and their 
children need not be inconsistent with the goal of moving parents to work. In fact. New Hope 
adds to a growing body of evidence that work-based support programs can increase parents’ 
work, earnings, and income and, in turn, can have beneficial effects on children — effects that 
translate into better performance in school. The annual cost of the program — at $5,300 per 
family (not per child) — is not trivial, but neither are its benefits. 



’P. A. Morris, A. C. Huston, G. J. Duncan, D. A. Crosby, and J. M. Bos, How Welfare and Work Policies 
Affect Children: A Synthesis of Research (New York: MDRC, 2001). 
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Chapter 1 

The New Hope Project and Evaluation 



For the past twenty years, public policies for poor families have increasingly empha- 
sized employment while reducing “welfare,” or public assistance, to families in which parents 
did not work for pay. Besides affecting parents’ employment and economic well-being, such 
policies are also likely to affect family life and children’s development. 

The New Hope Project — which was designed and implemented in Milwaukee, Wis- 
consin — offered an innovative and comprehensive approach to reduce poverty, reform wel- 
fare, and address the economic self-sufficiency of poor people who can work. New Hope con- 
sisted of four components: job search assistance, including referral to a wage-paying commu- 
nity service job when necessary; an earnings supplement to raise low-wage workers’ earned 
income above the poverty line; subsidized health insurance; and subsidized child care. The pro- 
gram was based on two principles: (1) that people who are willing to work full time should have 
the opportunity to do so and (2) that people who work full time should not be poor. 

New Hope was designed as a demonstration for a combination of work supports that 
could be replicable as government policy. The project was funded by a consortium of local, 
state, and national foundations and other organizations interested in work-based antipoverty pol- 
icy and by the State of Wisconsin and the federal government.' 

The New Hope Project was designed to provide information to policymakers. Would the 
program succeed in boosting employment, raising earned income, increasing economic security, 
reducing poverty, and lowering the use of public assistance? Would it affect family functioning 
and the lives of the children? Would it affect children’s long-term development? To answer these 
and other policy questions, evaluations of the impacts of the program on parents, families, and 
children were conducted two and five years after New Hope was initiated. The results of the two- 
year evaluation were published in an earlier report.^ Individuals’ participation was limited to three 
years. The present report documents the program’s effects five years after participants enrolled 
(that is, two years after an individual parent’s eligibility for the program ended).^ 



'See Appendix A for a list of organi 2 ^tions fiinding the New Hope Project. 

^Bos et al., 1999. 

^Readers who are primarily interested in New Hope’s history, designs, and operations should refer to the 
comprehensive report on those issues: Creating New Hope: Implementation of a Program to Reduce Poverty 
and Reform Welfare (Brock, Doolittle, Fellerath, and Wiseman, 1997). Prior publications also include The New 
Hope Offer: Participants in the New Hope Demonstration Discuss Work, Family, and Self-Sufficiency (Benoit, 

(continued) 
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This chapter introduces the New Hope program and its objectives and describes the de- 
sign of the New Hope Child and Family Study (CFS). 



Description of the Program 

The New Hope Project was conducted in two inner-city areas in Milwaukee. The pro- 
gram had only four eligibility requirements: that applicants live in one of the two targeted ser- 
vice areas, be age 18 or over, be willing and able to work at least 30 hours per week, and have 
a household income at or below 150 percent of the federally defined poverty level. Participa- 
tion in New Hope included the following benefits; 

• Job Access. Participants who were unemployed or who wanted to change 
jobs received individualized job search assistance. If participants could not 
find work in the regular job market after an eight- week job search, they could 
apply for a community service job (CSJ) in a nonprofit organization. These 
opportunities were also offered to participants who were between jobs or 
who were employed but not working the 30-hour minimum. The CSJs paid 
minimum wage and might be either full time or part time. 

• Earnings Supplements. New Hope offered monthly earnings supplements 
to participants who worked at least 30 hours per week but whose earnings 
left their household below 200 percent of the poverty line. CSJ wages and 
employment were counted toward the 30-hour requirement, and they also 
qualified a participant for the federal and Wisconsin Earned Income Tax 
Credits (EITCs). Combined with the EITC, New Hope’s earnings supple- 
ments raised most participants’ annual household income above the federal 
poverty threshold."* 

• Health Insurance. New Hope offered a health insurance plan to participants 
who worked at least 30 hours per week but were not covered by employers’ 



1996); fVho Got New Hope? (Wiseman, 1997); and An Early Look at Community Service Jobs in the New 
Hope Demonstration (Poglinco, Brash, and Granger, 1998). 

“’Participants’ income could be below the poverty line if they worked just 30 hours, but it would rise 
above the line as their hours increased. The exception was for very large households: Earnings supplements 
were adjusted upward for household size, up to a maximum of two adults and four children. New Hope’s 
other financial benefits — health insurance and child care — were extended to all eligible household 
members, regardless of household size. For more detail on how the financial benefits were calibrated, see 
Appendix C in Brock et al. (1997). As an example, in 1994, one wage-earner with two children would have 
received S68 per month in supplement payments; in 1996, however — given the expansion of the EITC and 
the fact that supplement payments are paid on top of EITC benefits — this same wage-earner would have 
received only S20 per month in supplement payments. 
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health insurance or Medicaid. Participants were required to contribute toward 
the health insurance premium on a sliding scale that took into account their 
income and household size; New Hope subsidized the remainder. 

• Child Care Assistance. New Hope offered financial assistance to cover 
child care expenses for children under age 13 when the participating parent 
worked at least 30 hours per week. Participants were asked to pay a portion 
of the cost, based on their income and household size; New Hope covered 
the remainder. For participants to qualify for New Hope subsidies, the child 
care had to be provided in state-licensed or county-certified homes or child 
care centers. 

• Staff Support. All participants were assigned to project representatives who 
could provide advice and information about employment (for example, help 
in finding a job), child care, or other topics (see Box 1.1). The program’s 
model emphasized respect and helpfulness in staff interactions with partici- 
pants. Indeed, a key finding fi-om a prior report was that many participants 
found the support and encouragement that they received fi-om staff to be use- 
ful and positive.^ 

Participants in New Hope who met the 30-hour work requirement could use any num- 
ber or combination of program benefits and services, depending on their needs. Eligibility for 
earnings supplements, health insurance, and child care assistance extended for three years after 
the date a participant entered the program (the date of random assignment). The time limits 
reflected funding constraints and were not considered integral to the program’s design. Rather, 
most of New Hope’s designers assumed that benefits would need to be permanently available 
if New Hope were to become ongoing policy. 



The Conceptual Model Guiding This Evaluation 

New Hope was designed to have direct effects on adult participants’ employment and 
use of benefits and services, including health insurance and licensed child care. These direct 
impacts could influence the well-being of participating adults and their families indirectly. The 
conceptual fiamework represents an evaluation model using a “theory of change” — that is, 
using theory and prior research to propose the intervening processes by which change can take 



^Brock, Doolittle, Fellerath, and Wiseman, 1997. 
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Box 1.1 



New Hope’s Project Representatives Provided 
Crucial Practical and Emotional Help for Working-Poor Parents 

A random sample of 44 families from the Child and Family Study (CFS) sample — 
half from the program group and half from the control group — took part in the New Hope 
Ethnographic Study (NHES), involving frequent interviews and observations over three years. 
Many New Hope participants described their positive experiences with proactive, emotionally 
supportive project representatives and the cmcial help they received in getting practical in- 
formation and assistance. In contrast, very few parents in the control group sample said simi- 
lar things about the service providers that they encountered in other social agencies. 

Many of New Hope’s project representatives provided emotional support and en- 
couragement to participants facing difficult situations. Frida, a single Latina mother of two 
children, said that she “needed a lot” and described how the project representatives gave her 
help. She said: “They were like a forward push for me. The best thing I received from them 
was their moral support.” 

Rose, an African-American single mother of two boys, really liked the project repre- 
sentatives because they gave needed advice and emotional support. Often when she was feel- 
ing down about herself and her situation, her project representative would point out the posi- 
tive things that Rose had already accomplished, reminding her of her success in working and 
taking care of her kids. Rose said that she often felt better about herself after she talked with 
people at New Hope. 

New Hope’s project representatives also provided invaluable practical information to 
participating families. They informed participants about the Earned Income Tax Credit and pro- 
vided advice on where to get education, skills training and employment, workshops on home 
purchase, and other asset development strategies. Ahcia, a married Latina mother of a teenage 
girl, described her experience as transforming: “I was accepted in New Hope, and everything 
changed for me.” New Hope encouraged her to go to a local technical college and study Eng- 
lish. Before that time, she was afiaid to speak it, and she felt acomplejada (“inferior”). She said 
that New Hope “built up my morale and my self-esteem.” Project representatives also invited 
her to workshops that provided information about the program and about how to find a job in 
her field. According to Ahcia, the most important impact of New Hope was that it improved her 
knowledge about community resources and how to use them successfully. 

Finally — for at least some participants — the practical support of New Hope’s pro- 
ject representatives extended to the transition out of the program after their three years of eli- 
gibility ended. Inez, a married Latina mother of two boys, was very grateful for her project 
representative’s help in getting information about child care subsidies from the county when 
her eligibility for New Hope was ending: “[My representative] was the one who told me 
where to go. She helped me out a lot.” 

Although the New Hope Project and evaluation focus on the program’s benefits and 
their impacts on employment and other outcomes, the ethnographic and implementation stud- 
ies suggest that the concerned, continuous personal relationships that many clients had with 
project representatives were often remembered by participants as being among New Hope’s 
most important influences on their lives. 
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place.® Figure 1.1 summarizes the model that has guided the evaluation of New Hope’s impacts 
on children and families: 

• The New Hope Offer. The New Hope program offered employment ser- 
vices, wage supplements, health care assistance, child care assistance, staff 
support, and community service jobs (CSJs). 

• Employment and Earnings. The increased use of New Hope’s supports 
was intended to increase participants’ work hours, earnings, and income and 
to reduce their reliance on public assistance. 

• Adults’ Well-Being. Increased employment — while perhaps stressful dur- 
ing some period of transition — was expected to improve adults’ material 
and psychological well-being. Some of this improvement would occur be- 
cause employment is socially desirable and unemployment and welfare are 
stigmatized. In addition, adults’ well-being was expected to increase because 
of greater income and reduced poverty.’ 

• Children’s Contexts. Changes in employment, income, and parents’ well- 
being were likely to affect home environments and the context in which chil- 
dren spent their time when they were away from their parents. Changes in the 
number or nature of the interactions between parents and children might 
occur.* Finally, employment, income, and the distinctiveness of the New Hope 
child care subsidy were all expected to affect the nature and number of child 
care experiences for younger children as well as children’s participation in 
stmctured out-of-school activities, particularly for older children and adoles- 
cents. 

• Children’s Outcomes. Changes in the home environment and in experi- 
ences away from home could affect children’s intellectual skills, psychologi- 
cal well-being, social skills, and health. Improvements in these areas could 
lead to fewer behavior problems and less deviant behavior as children move 
into adolescence. 



®Fulbright-Anderson, Kubisch, and Connell, 1998; Kubisch, Weiss, Schor, and Connell, 1995. 

’Duncan, Brooks-Gunn, and Klebanov, 1994. 

*McLoyd, Jayaratne, Ceballo, and Borquez, 1994; Conger et al., 1992; Garrett, Ng’andu, and Perron, 1994. 
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The New Hope Project 
Figure l.I 
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ETHNIC GROUP, CULTURE, FAMILY BACKGROUND 



The Evaluation’s Design, Activities, and Data Sources 

Random Assignment Design 

The New Hope evaluation was built around an experimental design. Applicants who 
met the eligibility criteria were randomly assigned either to a program group that could partici- 
pate in New Hope or to a control group that could not. By comparing the outcomes of the two 
groups over time, it is possible to distinguish the effects specific to New Hope from changes 
that might have occurred for other reasons, because the random assignment process ensures 
that the characteristics, backgrounds, and motivation levels of program and control group 
members did not differ systematically at the beginning of the study. After random assignment, 
the only systematic difference between the program group and the control group was that the 
former had access to New Hope. Therefore, any differences between the adults or children in 
the two groups can be attributed to the New Hope intervention. 

The Child and Family Study Sample 

The New Hope Project enrolled 1,362 low-income adults drawn from two inner-city 
areas in Milwaukee. The two-year impacts of the program on this total sample are described in 
an earlier report.’ A brief report of impacts after five years will be issued separately. 

The current report presents findings for a subgroup of the total sample — the Child 
and Family Study (CFS) sample — that was selected in order to evaluate the program’s effects 
on children and families as well as on employment and income. The CFS sample includes all 
745 adult sample members who had one or more children between the ages of 1 year, 0 
months, and 10 years, 11 months, at the time of random assignment (55 percent of the total 
sample)." If a family had more than one child in that age range, two children were identified as 
“focal children.”'^ The evaluation includes a total of 1,140 focal children; a limited amount of 
information was collected about other children in the family. 

Random assignment of the total New Hope sample began in August 1994 and ended in 
December 1995. All sample members were volunteers who applied for the program; applicants 

’Five cases were dropped from the total sample because of missing baseline data, resulting in a total sam- 
ple of 1,357. For the two-year impacts of New Hope, see Bos et al. (1999). 

'’The earlier report also presents findings for the CFS sample. 

"The CFS sample excludes 67 Asian- American families — most of whom are Southeast Asian refugees 
— because of language barriers and because many of the measurement instruments are culturally inappropriate 
for them. 

'^If there were more than two ehgible children, the focal children were randomly selected with the restric- 
tion that opposite-sex siblings were given preference over same-sex siblings. 
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were randomly assigned either to the program group or to the control group. The parents in the 
CFS sample were in many respects similar to those in other studies in which samples were drawn 
from individuals receiving welfare. When they applied for New Hope, over half were not em- 
ployed, and about 80 percent were receiving Aid to Families with Dependent Children (AFDC), 
general assistance, food stamps, and/or Medicaid. The majority had never been married. Slightly 
over 10 percent were married and were living with their spouse, and almost half had three or more 
children. Slightly over half are African-American, and over one-quarter are Hispanic. 

Data Sources 

Information used in the New Hope evaluation came from five sources: 

1. Information about the members of the research sample at enrollment, or 
“baseline,” came from an enrollment form that was completed prior to ran- 
dom assignment. 

2. Administrative records included unemployment insurance (UI) earnings re- 
cords measuring quarterly earnings and employment; public assistance 
benefit records documenting welfare payments, food stamps, and Medicaid 
benefits; tax records providing information on the receipt of Earned Income 
Credits (EITCs); and New Hope administrative records providing informa- 
tion about receipt of New Hope benefits. 

3. In-person surveys with parents and children were conducted two years and 
five years after parents were randomly assigned to the program or control 
group. The survey measured receipt of non-New Hope services; many eco- 
nomic outcomes (for example, hours of work, hourly wages, and the type of 
Jobs held); family fimctioning (including parental well-being and parent- 
child relations); children’s participation in child care and out-of-school ac- 
tivities; and children’s behavior and development. 

4. For school-age children, surveys were mailed to teachers to obtain reports 
of children’s school performance and social behavior (both positive and 
negative). 

5. In 1998, the research team began the New Hope Ethnographic Study 
(NHES) of 44 families from the CFS sample. The study continued for 
three years and includes members of both the program and the control 



’^Details of the random assignment process are presented in Bos et al. (1999). 
‘‘'Weisner, Gibson, Lowe, and Romich, 2002. 
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groups. Figure 1.2 presents the time line for the New Hope evaluation and 
the timing of assessments. 



Results of the Evaluation Two Years After Enrollment 

New Hope was open to all adults who met three basic eligibility requirements: being 
age 1 8 or older, having an income below 1 50 percent of the poverty threshold, and being will- 
ing to work full time. Hence, not all participants had children. In the first evaluation of the pro- 
gram — conducted two years after sample members were randomly assigned — the total sam- 
ple received a core survey assessing employment and economic outcomes. In addition, the 
CFS sample received measures of parental well-being and of children’s experiences and devel- 
opmental outcomes. 

At the two-year evaluation, when focal children were between ages 3 and 12 years old, 
parents in the New Hope program group had significantly higher rates of employment and 
higher earnings than did those in the control group. CFS program group members earned 
$1 5,305 during the two years of follow-up, compared with $13,846 for their counterparts in the 
control group. Although the impacts on earnings were concentrated among the parents who had 
not been employed full time at baseline, the differences in impacts for this group and for the 
group employed full time at baseline were not statistically significant.'* 

There were some positive impacts on parents’ psychological well-being, but not on 
measures of mental health. New Hope participants reported less stress, fewer financial worries, 
and increased feelings of agency — the belief that they could take action and achieve then- 
goals. On a possibly negative note, parents in New Hope reported significantly more time pres- 
sures, which may have been related to their increased work efforts. 

New Hope had strong effects on children’s experiences outside their homes but little 
measured impact on the home environment or on parent-child relations. Most striking is that 
New Hope’s child care subsidies made formal center-based and after-school child care pro- 
grams more affordable and stimulated their use by program group families. Both preschool and 
elementary-school-aged children in New Hope families were more likely to attend child care 
centers and to participate in after-school programs. Older children, ages 9 to 12 years, also par- 
ticipated more in such structured activities as lessons, sports, clubs, and religious groups. These 
experiences could have contributed to school achievement and to children’s social skills, and 
they may have provided supervision when parents were away at work. 



'*See Bos et al. (1999) for results. 
'*Boset al., 1999. 
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Perhaps most important, children in program group families were making better aca- 
demic progress and displayed more positive social behavior than children in control group fami- 
lies. All these differences occurred primarily for boys; the patterns for girls were less consistent 
and more mixed. Teachers rated program group children of both genders higher on academic 
performance and positive social behavior than they rated control group children. They rated 
New Hope boys higher on classroom behavior (for example, study skills, ability to attend) and 
lower on behavior problems and need for disciplinary actions in the classroom than they rated 
control group boys. New Hope also had positive impacts on boys’ own expectations about edu- 
cational and occupational attainment.’’ 

From a policy perspective, the impacts on boys are important because boys are more 
vulnerable than girls to school failure and delinquency as they reach adolescence. Several pos- 
sible reasons for the greater impact on boys were suggested. New Hope boys were more likely 
than girls to be enrolled in after-school child care and stmctured activities, so more of them may 
have benefited from these experiences. Ethnographic interviews suggest that parents were wor- 
ried about their boys’ becoming involved in delinquent activity and may have invested more 
resources in ensuring that boys had alternatives to hanging out with unsupervised peers after 
school. There was not evidence that girls were assuming more household responsibilities than 
boys did while their mothers worked, but boys and girls may have responded differently to the 
role models provided by their parents. About 90 percent of the parents in the CFS sample are 
women; thus, girls may have used their participating parents’ jobs as examples of their own fu- 
ture more than boys did. The realities of the low-wage employment world for women may have 
become apparent to children in these families. 



Why New Hope Might Have Lasting Effects 

New Hope’s designers conceived of the program as a set of work supports that would 
be in place as long as individuals needed them. Although the demonstration program was not 
designed to evaluate the effects of time limits, it limited any individual’s eligibility to three 
years because of financial constraints. This report presents an evaluation conducted five years 
after parents entered the New Hope study (that is, two years after the end of eligibility). Hence, 
this evaluation tests the possibility that the changes that were observed during eligibility en- 
dured after families no longer received benefits. Even though New Hope was not intended to 
demonstrate a time-limited policy, there are several reasons to expect that the three-year period 
of benefits might have had lasting effects on parents, children, and family life. 



’’Huston et al., 2001. 
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First, if parents gained job experience and confidence in their ability to earn a living, 
some of the employment and income impacts of New Hope might have continued after benefits 
were discontinued. This possibility is particularly strong, because the EITC continued to be 
available as an important supplement to parents’ earnings and, in fact, increased in value during 
the period fi-om 1995 to 2000. Evidence fi'om the ethnographic work, for example, suggests that 
families choose lump sum refund payments to provide a form of savings and to purchase expen- 
sive items (cars, furniture) or to pay down debt (mortgage, credit cards). Lump sum payments 
fi'om EITCs may have continued to benefit families with sufficient earnings.'* There is evidence 
fi'om several policy experiments that employment programs offering earnings supplements pro- 
duced improved achievement among children.'® 

Second, children’s experiences in formal child care and structured out-of-school activi- 
ties may have occurred during “sensitive developmental periods” during which experiences 
have formative effects that endure through later periods. For example, if formal child care pro- 
vides some basic preacademic skills for preschool children, the children may enter school with 
some academic preparation that sets off a trajectory of success. Teachers may perceive such 
children as being more skilled and may provide more opportunities for learning. Organized af- 
ter-school programs during the early school years may contribute to children’s academic and 
social skills as they progress into the later school years. If structured out-of-school experiences 
provide supervision and social skills, then New Hope’s children may continue to pursue some 
of the activities in which they participated during the three-year period of benefits. 

Advantages that accrued during New Hope’s benefit period may lead to an upward spi- 
ral in children’s development. Initial experiences may change the child’s behavior or capabili- 
ties; as a result, the child generates different types of input fi'om the environment; that input, in 
turn, maintains or increases the behavior or skill involved. In this model, treatment-induced 
changes in the child’s behavior “drive” the context, either by eliciting particular reactions fi'om 
the people around the child or by leading the child to seek out different contexts.^" 

Finally, the changed contexts brought about by New Hope may have continued after the 
program ended. Some of the effects on parents’ employment, income, and family routines as 
well as on child care, activities, and school may have enduredT Parents may have acquired skills 
in negotiating bureaucracies and finding community resources for their children. For example, 
program group families may have continued to use center-based child care in preference to 
home-based care, and they may have been able to find subsidized care in the community. Help 
fi'om project representatives in finding job training and educational programs may have led 



'*Romich and Weisner, 2000. 

'®Morris et al., 2001. 

^“Entwisle, Alexander, and Olson, 1997; Scan and McCartney, 1983. 



some participants to pursue additional education. These changed contexts brought about by 
New Hope could have maintained changes in family life and children’s behavior. 



New Hope and the Policy Context from 1995 to 2000 

The New Hope evaluation occurred during a period of rapid changes in federal, state, 
and local policies affecting parents who are poor. Major events included welfare reforms that 
focus directly on employment as well as efforts to make low-wage work “pay” by expanding 
federal and state EITCs. These changes occurred during a time of exceptionally low unem- 
ployment in Milwaukee (the rate fluctuated between 5.0 percent and 3.6 percent), making it 
generally easier for people to get and hold jobs as employers adjusted to a tighter labor supply. 
Changes in the federal rules and flmding for child care assistance and Medicaid were designed 
to decouple these programs from cash assistance — that is, to make them available to income- 
eligible parents who were not receiving cash assistance through welfare programs. States were 
given a great deal of latitude to design and implement their own welfare programs and to use 
welfare flmds for child care and other related services. 

These economic and policy changes — which affected the supports available to members 
of both the program and the control groups — “raised the bar” for showing the effects of New 
Hope by diminishing the difference between what the program offered and what was available 
outside New Hope. It was therefore more difficult for New Hope to create a net difference. 

Changes in the Welfare System 

New Hope began enrollment in 1994, and program participation continued into 1998. In 
1 996, the system providing cash supports to low-income parents in the United States changed 
dramatically with the passage of the Personal Responsibility and Work Opportunity Reconcilia- 
tion Act (PRWORA). Entitlements to Aid to Families with Dependent Children (AFDC) were 
ended, and a new system of Temporary Assistance for Needy Families (TANF) was introduced. 
One of the major goals of the law was to move families from “welfare to work,” and states were 
given broad latitude in setting up programs to accomplish this goal.^‘ 

Even before passage of the federal legislation, the State of Wisconsin was a leader in 
phasing out AFDC in favor of a program emphasizing employment. In 1994, Wisconsin’s gov- 
ernor announced that AFDC would end in the state. A welfare reform known as Pay for Per- 
formance, which tied receipt of AFDC benefits to work, began in 1996. After the federal wel- 
fare reform legislation was passed in 1996, all prior welfare programs were replaced by Wis- 
consin Works (W-2), which was implemented statewide in 1997 and 1998. Under this program, 

^‘For summary of PRWORA and its implementation, see Greenberg et al. (2002). 
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all recipients and applicants for TANF are required to complete Personal Responsibility Con- 
tracts and Employability Plans that specify the employment activities that will be required as a 
condition for receiving assistance.^^ 

Under W-2, an applicant for TANF who is deemed employable does not receive cash 
grants but does get assistance in finding a job or assistance with different types of transitional 
employment. Welfare recipients can “earn” their welfare grant by working in a community ser- 
vice Job, but unlike the New Hope program, these positions do not allow people to earn wages 
or to qualify for EITCs. Rather, these Jobs are a means to “work off’ one’s welfare grant. Indi- 
viduals who have barriers to employment receive cash grants (ranging from $518 per month in 
1996 to $673 per month in 2000, with no adjustment for family size). 

These policy changes had dramatic effects on Wisconsin’s caseloads and the allocation of 
funds. In Milwaukee, the number of people receiving AFDC dropped from over 35,000 in 1995 to 
21,400 in 1997 — a decline of 41 percent — before W-2 was implemented.^^ By 2000, only 5,427 
adults were receiving W-2 cash assistance, and another 3,180 were receiving W-2 case manage- 
ment.^"* Such dramatic changes in the availability of cash assistance and work requirements for all 
persons in the TANF system are likely to have provided strong employment incentives for New 
Hope’s sample members, whether in the program group or the control group. 

Changes in Earnings Suppiements for Working-Poor Aduits 

During the same period that welfare reform took place, federal and state policies in- 
creased work supports for low-income families. The goals of these policies are to “make work 
pay” and to provide basic supports (for example, child care assistance, medical protection) to 
working-poor families. Earnings supplements in the form of the federal Earned Income Tax 
Credit (EITC) — first established in 1975 — were increased considerably in the 1990s. During 
the New Hope study period, the maximum federal benefit changed from $3,110 in 1995 to 
$3,888 in 2000, and the State of Wisconsin offered a benefit that was approximately 25 percent 
of the federal benefit, depending on family size. 

New Hope’s earnings supplement “topped up” earnings plus any federal and state 
EITCs. The amount of the supplement was structured to provide an incentive for increased earn- 
ings, up to a targeted annual income of either $30,000 or 200 percent of the poverty level, 
whichever was higher for a given family type. All workers can apply for federal and state 



^^State of Wisconsin, Department of Workforce Development, 2001. 

^^Pawasarat, 2000. 

^‘‘State of Wisconsin, Department of Workforce Development, 2002a. The 1995 and 2000 figures are not 
entirely comparable because all AFDC child-only cases were transferred out of the W-2 system (Ehrle, Seed- 
felt, Snyder, and McMahon, 2001). 
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EITCs. Therefore, when the federal EITC was enhanced, the relative importance of the New 
Hope supplement diminished for all participants. For example, in November 1994, a single 
wage-earner with two children (the modal family structure of adults who enrolled in New Hope) 
and with gross wages of $12,000 drew combined federal and state EITCs of $2,856 and a New 
Hope supplement of $816; in September 1997, the same earner drew EITCs of $3,960 and no 
New Hope supplement. On the one hand, the changes in the EITC might have reduced the ef- 
fect of the New Hope supplement on total income and work effort for people with children. On 
the other hand. New Hope staff regularly informed program participants about the EITCs; 
hence, more program group members may have filed the tax returns necessary to receive them. 

Changes in Child Care Assistance for Low-Income Families 

Prior to 1996, federal subsidies for child care were in four different fimding streams, three 
of which were designated for people receiving AFDC, leaving AFDC, or in danger of entering 
AFDC. The fourth program was intended for low-income families regardless of welfare status. 

New Hope’s child care supplement was more advantageous than most other child care 
subsidies available to the working poor or to welfare recipients in Milwaukee. Other subsidies 
were not available to all eligible parents, especially if they were not on welfare, whereas New 
Hope’s child care subsidy was available to everyone who worked full time. The subsidy phased 
out gradually with increased earnings, but the upper limit was set higher than most other subsidy 
programs, at $2,500 per month (an annual equivalent of $30,000) in 1994. After a small copay- 
ment, New Hope paid the market cost of any licensed or certified child care center or home care 
arrangement. New Hope paid providers directly and promptly, in contrast to many public systems 
that have long delays. New Hope project representatives provided information about child care 
facilities to parents who requested it. The paperwork and eligibility determination, although done 
monthly, were part of the overall process of certifying that the parent had worked 30 hours per 
week. Other subsidy programs sometimes required separate determinations of eligibility for child 
care assistance. Parents often needed to apply for different subsidy programs as their situation 
changed (for example, if they left AFDC for work or they reached a time limit for benefits related 
to the transition fi’om AFDC). In addition, child care funding — particularly for working-poor 
people who had not been on AFDC — was not available at a level that could meet the demand. 
The administrative complexities and limited funding meant that many people who were eligible 
for child care supplements outside of New Hope did not get them. 

In 1996, with the passage of federal welfare legislation, the pre-1996 federal subsidies 
were combined into one Child Care Development Fund.^’ Wisconsin took this opportunity to 
make several changes regarding child care funding, and the net effect may have decreased New 

^^Fuller, Cagan, Caspary, and Gauthier, 2002; Gennetian et al., 2002b. 
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Hope’s comparative advantage, but the income eligibility limit was still considerably higher for 
New Hope than for the state. Although federal law allows states to set eligibility at 85 percent of 
the state median income (SMI), Wisconsin set it at 53 percent. Hence, in the late 1 990s, the an- 
nual income eligibility limit was $21,996, compared with $30,000 for New Hope.^^ Wisconsin 
significantly supplemented the federal dollars with the expressed goal of making one seamless 
subsidy system available to all eligible families. It also increased the maximum reimbursement 
rates for placements in both centers and family child care homes, but a larger copayment was 
required for center care than for home-based care. By the year 2000, expenditures on child care 
had risen to approximately $217 million (fi-om about $64 million in 1996).^^ Given the limited 
funding available prior to 1997 for persons not on AFDC (or on transitional benefits), these 
changes probably increased the number of parents receiving a subsidy and also the level of the 
subsidy. On a national level, however, only 15 percent to 20 percent of federally eligible parents 
received subsidies,^* so it is likely that many eligible working parents still had difficulty obtain- 
ing a subsidy, particularly if they were outside the welfare system. 

Changes in Medical Insurance 

Prior to 1996, health insurance via Medicaid was provided to all AFDC recipients and 
to some other low-income families. Subsequently, transitional Medicaid eligibility was avail- 
able to parents leaving TANF, but federal law provided that Medicaid was not tied to cash assis- 
tance. Wisconsin introduced a second program, BadgerCare, that offers medical insurance to 
uninsured families who have children. Although the official income eligibility cutoff is 1 85 per- 
cent of poverty, most enrolled families have incomes below 150 percent of poverty.^® 

New Hope offered its participants the same health maintenance organization (HMO) 
that Milwaukee County Medicaid recipients used, so there was no difference between the New 
Hope plan and the one available to all welfare recipients prior to 1997 and to most poor families 
after that. New Hope’s advantage was that it provided affordable health care to people not on 
welfare before 1997 and was more affordable for most participants than plans provided by pri- 
vate employers. 

As this discussion of the policy context makes clear New Hope’s benefits and services, 
viewed individually, have features in common with other programs and public policies. If any- 
thing, this is becoming more true over time in Wisconsin. The dramatic changes in federal and 
state welfare, child care, and health policies over the course of the New Hope evaluation are 



^^Information about the Wisconsin system is from Blau and Tekin (2001). 

^^State of Wisconsin, Department of Workforce Development, 2002b. 

^*Layzer and Collins, 2002. 

^’Wisconsin BadgerCare Web site: http://www.dhfs.state.wi.us/badgercare/general.htm. 



likely to have affected families in the control group as well as in the program group. Thus, the 
impacts of New Hope — even with its strong package of benefits and services — may be un- 
derestimated because of the effects of policy changes on all sample members. It should also be 
noted that New Hope was created with the intention of influencing public policy in the same 
areas, so that the expansion of child care subsidies and high-quality, low-cost health insurance 
(Family Shares and BadgerCare) were viewed as the desired outcomes. (Whatever the apparent 
similarities between W-2 and New Hope, Box 1.2 describes some differences in how partici- 
pants experienced the two programs.) 



The Organization of This Report 

This report follows the model depicted in Figure 1.1. Chapter 2 provides a brief sum- 
mary of New Hope’s operations. It presents the background characteristics of the research sam- 
ple and compares the program and control groups’ use of benefits and services. Chapter 3 pre- 
sents the program’s effects on such economic measures as employment, earnings income, and 
public assistance. Chapter 4 describes the effects on parents’ material and psychological well- 
being. In Chapter 5, impacts on contexts affecting children are presented. These include parent- 
ing and parent-child relationships, child care, and out-of-school activities. Chapter 6 presents 
data on how New Hope affected child outcomes, including reports from parents, teachers, and 
children. Chapter 7 presents conclusions about the overall impacts of New Hope and discusses 
their implications for public policy. 
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Box 1.2 

Clients Find Bureaucratic Complexity and Little Support 
at Wisconsin Works (W-2) 

The New Hope Ethnographic Study (NHES) highlights a sharp contrast between 
the atmosphere in the New Hope program (see Box 1.1) and the experiences that both 
program and control group participants had with Wisconsin Works (W-2). Almost none 
of the families in the ethnographic sample reported positive interactions with the W-2 
caseworkers or the benefits that were administered by the system. In the first year of the 
NHES, some families were ending their New Hope participation and were transitioning 
into the early stages of W-2. The NHES then followed and documented families’ experi- 
ences with W-2. Most families had two general complaints.* 

First, they reported fiustration with the bureaucratic complexity of the W-2 system 
— p^icularly noting the obstacles that were put in place during the application process, the 
low income thresholds for receiving aid, and the “hyper-surveillance” concerning families’ 
ongoing eligibility for benefits. Evelia, a single Latina mother of four children, fi-equently 
vented her annoyance about how difficult it seemed to qualify for benefits, because W-2’s 
low income thresholds required the termination of benefits. Evelia believed that administra- 
tive agencies made it difficult to get benefits in order to motivate people to go to work and 
stop using state cash assistance. She agreed that people should work, but she was unhappy 
with what she perceived to be deliberate barriers for those families who needed some type 
of help. Moreover, when she did apply for such work supports as child care, she experi- 
enced maddening delays and bureaucratic obstacles. It took her an entire week of interviews 
and meetings to get subsidized child care for her daughter — forcing her to miss much- 
needed time at work, where she was paid hourly. 

Second, families reported a lack of emotional and practical support fi’om case- 
workers in the W-2 system in Milwaukee County. When asked to compare her experi- 
ences with New Hope’s project representatives and with W-2’s caseworkers, Wendy, a 
married Affican-American mother of three boys, stated: “[New Hope representatives] 
were so caring ... [at W-2] they don’t care. . . . They didn’t treat us as, you know, people. 
They treated us as though we were nothing.” She continued: “[W-2 offered] no kind of 

education or nothing Just whatever was going on in our household, that was it. New 

Hope had so much out there, any kind of job workshops or whatever. They would call us 
and let us know.” Wendy felt that New Hope’s project representatives were looking out 
for her, that they were actively searching for programs and opportunities that might bene- 
fit her. She believed that W-2 workers treated her with little respect. She said: “Once you 
sit there for a while, its like . . . there was no kind of communication, no kind of connec- 
tion, nothing. No respect. They would make you feel like you lacked self-esteem. Even if 
you didn’t, they would make you feel that way, because they were snotty.” 

Complaints like these in Milwaukee County have not gone unnoticed by state leg- 
islators, W-2 administrators, and the media. The volume of complaints prompted the state to 
audit the private agencies that administer W-2 in Milwaukee County in the spring of 2001 
and to reconmend a number of changes to improve services.^ These badly needed reforms 
met stiff political resistance fi-om some of the more conservative state legislators, and, as a 
result, reforms that were scheduled to begin in January 2002 were delayed until July 2002.^ 



*Gibson and Weisner, 2002. 

|State of Wisconsin, Department of Workforce Development, 2001. 
^Schultze, 2001. 
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Chapter 2 



Characteristics of the Study Sample and 
Participants’ Use of Benefits and Services 



The two-year follow-up report of the New Hope Project was based on data that were 
gathered when participants were in the midst of their three-year eligibility periods.' The five- 
year follow-up gathered data well after the end of New Hope. This chapter begins by describing 
the characteristics of the Child and Family Study (CFS) sample at five years, and then it details 
participants’ use of New Hope services throughout their entire three-year eligibility periods. 
Administrative records and the five-year survey also provide data on the effects of the end of 
New Hope eligibility. Finally, the chapter compares the receipt of health and child care benefits 
by program and control group members five years after random assignment, and it presents the 
costs of New Hope per program group member. 

The Child and Family Study (CFS) Sample 

The Samples Used in This Report 

Participants in the New Hope experiment were recruited over a period beginning in 
August 1994 and ending in December 1995. The total New Hope sample includes 1,357 partici- 
pants who were randomly assigned during that period.^ The Child and Family Study (CFS) 
sample was selected fi"om the total sample in order to examine the program’s effects on families 
and children. Figure 2.1 illustrates how the samples presented in this report were derived fi"om 
the total New Hope sample. 

The CFS sample includes all 745 members of the total sample who had one or more 
children between the ages of 1 year, 0 months, and 10 years, 1 1 months, at the time of random 
assignment. Surveys were administered to CFS families at two years and at five years after ran- 
dom assignment. The analyses presented in this report focus'oh the survey sample of 561 CFS 
families who responded to the five-year follow-up survey; however, some impacts are presented 
for the entire set of 745 families in the CFS sample. Up to two children in each CFS family 
were identified as “focal children’’ to be studied.^ Surveys were administered to focal children at 



‘Bos etal., 1999. 

^Five cases lacked baseline data and were dropped from the original total sample of 1,362. 

^If a family had more than two eligible children, the focal children were randomly selected, with the re- 
striction that opposite-sex siblings were given preference over same-sex siblings. 
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The New Hope Project 



Figure 2.1 

Samples Used in This Report 




NOTES: “Originally, 1,362 sample members were randomly assigned, but 5 cases were dropped due to missing 
baseline data. 

*^For parent-reported child outcomes, the sample size is slightly larger because parents may have reported 
about children who did not complete the child survey. 

‘^This only includes one teacher per child. In some cases, multiple teachers responded for the same child. 
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the same time as surveys were administered to their parents. The child-reported outcomes pre- 
sented in this report are based on interviews with 840 children who responded to the five-year 
survey. These children make up the child outcomes sample and were between the ages of 6 and 
16 at the time of the survey. In addition, a mail survey was sent to teachers of children whose 
parents gave permission; teacher-reported outcomes are based on the reports of 547 teachers in 
this teacher survey sample!^ Finally, the analysis also draws data from the three-year New Hope 
Ethnographic Study (NHES) of 44 CFS families that was begun in the program’s second year 
(see Appendix B). 

A discussion of survey response rates and survey nonresponse bias for the five-year sur- 
vey samples is presented in Appendix C. The response rate for parents was 75.3 percent; for 
children, it was 72.3 percent; and for teachers, it was 63.2 percent. A comparison of the 561 re- 
spondents to the five-year survey and all 745 CFS families provides reassuringly little evidence 
of nonresponse bias. 

Characteristics of the CFS Sample 

Most of the 745 CFS families responded to the surveys at both of the follow-up points, 
but some families responded to one survey but not to the other.^ As a result, the sample used in 
the two-year report differs slightly from the sample used to generate five-year results. For the 
most part, the five-year results are based on the sample of families who responded to the five- 
year survey, but because many of the outcomes presented in this report are a follow-up to the 
two-year findings, CFS sample members who responded to the two-year survey are compared 
with those who responded to the five-year survey. 

One way to check for differences between the two samples is to compare their back- 
ground characteristics — all of which were measured at the time of random assignment (base- 
line). A lack of differences in background characteristics for the two samples generates confi- 
dence that program effects estimated at the two points can legitimately be compared, to reveal 
trends over time, even though the samples were slightly different at the two points in time. 

Table 2.1 compares the two-year and five-year survey samples on a number of back- 
ground characteristics that were measured when parents first applied for New Hope. The two 
samples are very similar on these baseline characteristics. The majority of families were headed 
by a single female parent (61.3 percent were never married, and 17.1 percent were separated. 



‘’in some cases, more than one teacher responded for a child. The report presents results for only one 
teacher per child. 

^Some 486 families responded to both surveys; 91 families responded to the two-year but not the five-year 
survey; and 76 families responded to the five-year but not the two-year survey. 
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The New Hope Project 
Table 2.1 



Selected Baseline Characteristics and Employment History of the CFS Sample, 

by Survey Cohort 





CFS" 


CFS" 




Five-Year 


Two-Year 


Characteristic 


Survey 


Survey 



Selected characteristics from Background Information Form 



Demographic characteristics 



Gender (%) 


Female 


91.4 


92.2 


Male 


8.6 


7.8 


Age (%) 


18-19 


5.0 


4.3 


20-24 


27.1 


28.3 


25-34 


48.8 


49.5 


35-44 


17.1 


16.1 


45-54 


1.4 


1.4 


55 or over 


0.5 


0.3 


Average age (years) 


29.4 


29.2 


Race/ethnicity (%) 


African-American, non-Hispanic 


55.6 


55.7 


Hispanic 


28.3 


27.4 


White, non-Hispanic 


13.0 


13.5 


Native American/Alaskan Native 


3.0 


3.3 


Household status 
Shares household with*^ (%) 


Spouse 


10.7 


9.2 


Girlfriend/boyfriend 


5.2 


5.7 


Children (own or partner's) 


96.3 


96.5 


Others 


16.3 


16.0 


Marital status (%) 


Never married 


61.3 


62.9 


Married, living with spouse 


11.1 


10.0 


Married, living apart 


10.5 


10.1 


Separated, divorced, or widowed 


17.1 


17.5 


Number of children in household‘s (%) 


None 


0.0 


0.0 


1 


24.1 


24.5 


2 


28.7 


29.5 


3 or more 


47.2 


46.0 



(continued) 
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Table 2.1 (continued) 



Characteristic 


CFS“ 

Five-Year 

Survey 


CFS“ 

Two-Year 

Survey 


Age of youngest Child‘S (%) 


2 or under 


48.0 


50.3 


3-5 


30.3 


28.7 


6 or over 


21.7 


21.0 


Age of child (%) 


1 -3 years 


28.2 


28.5 


4-10 years 


40.5 


39.1 


Both 


31.4 


32.5 


Labor force status 


Ever employed full time (%) 


83.4 


82.1 


Approximate earnings in past 12 months (%) 


None 


36.9 


37.3 


$l-$999 


16.4 


14.9 


$1,000-4,999 


23.0 


24.5 


$5,000-89,999 


13.2 


14.1 


$10,000-814,999 


7.3 


6.4 


$15,000 or above 


3.2 


2.8 


Current employment status (%) 


Employed 


38.7 


37.7 


Not employed 


55.6 


56.4 


Missing 


5.7 


5.9 


Among those currently employed 


Average hourly wage ($) 


6.35 


6.29 


Average hours worked per week (%) 


1-29 


22.4 


21.6 


30 or more 


77.6 


78.4 


Public assistance status 

Currently receiving AFDC, General Assistance, 

food stamps, or Medicaid (%) 


Any type 


81.1 


82.3 


AFDC 


69.5 


70.8 


General Assistance 


0.9 


0.7 


Food stamps 


77.4 


77.4 


Medicaid 


75.6 


76.0 


Total prior AFDC/General Assistance cash assistance® (%) 


None 


14.6 


13.5 


Less than 2 years 


26.4 


26.0 


2 years or more but less than 5 years 


25.9 


26.9 


5 years or more 


33.0 


33.5 



(continued) 
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Table 2.1 (continued) 





CFS" 


CFS" 




Five-Year 


Two-Year 


Characteristic 


Survey 


Survey 


Resided as a child in a household receiving AFDC (%) 


34.8 


35.1 


Educational status 






Received high school diploma or GED^(%) 


60.6 


61.8 


Highest grade completed in school (average) 


1 1.2 


11.2 


Currently enrolled in any type of education or training (%) 


37.8 


37.3 


Other factors related to obtaining/retaining employment 
Has access to a car (%) 


43.9 


43.8 


Ever arrested for anything since 1 6th birthday (%) 


18.9 


18.6 


Number of moves in past 2 years (%) 






None 


4.3 


28.3 


1 


30.1 


28.7 


2 or more 


28.5 


39.1 


Missing 


37.1 


4.0 


Employment history from Priyate Opinion Suryey 






Client-reported employment history 

Number of full-time jobs (30 hours or more a week) 

held in past 5 years (%) 






None 


21.3 


21.6 


1 


29.6 


28.2 


2 or 3 


37.6 


38.5 


4 or more 


11.6 


11.7 


When unemployed, length of time it took to find new work (%) 






1 month or less 


29.6 


28.6 


2-6 months 


37.2 


37.1 


More than 6 months 


13.9 


14.6 


Don't know 


19.2 


19.7 


Sample size 


561 


576 



(continued) 
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Table 2.1 (continued) 

SOURCES: MDRC calculations from Background Information Forms (BIFs) and Private Opinion Surveys 
(POS) for Child and Family Study (CFS) sample members who responded to either the two- or five-year 
survey. 

NOTES: Except for two BIF items, the nonresponse rate for all specific characteristics was less than 1 
percent, and therefore these missings were excluded from the calculations. For the two characteristics, for 
which the nonresponse rate ranged from 5 percent to 7 percent for the full sample, the nonresponses are shown 
in the table as missings. Among the POS responders, missings for individual questions ranged from 0 percent 
to 14 percent. 

Distributions may not add to 1 00 percent because of rounding. 

Actual sample sizes for individual measures may vary as a result of missing data. 

Tests of statistical significance were not conducted for comparisons of the 24- and 60-month CFS 
samples. 

^he sample includes all New Hope sample members (except Asian and Pacific Islander families) whose 
household included at least one child aged 1 to 10 at the time of random assignment. 

^Because some sample members may be in more than one category, totals may not equal all categories 
summed. 

‘^Includes all dependents under age 1 8. 

‘^Includes all dependents under age 1 8. 

^This refers to the total number of months accumulated from at least one spell on an individual's own 
AFDC or GA case or the case of another adult in the household. 

^The GED credential is given to those who pass the GED test and is intended to signify knowledge of 
basic high school subjects. 



divorced, or widowed), and most of these women were unemployed (55.6 percent) and 
receiving public assistance (81.1 percent). Their average age at baseline was 29 years. All racial 
and ethnic groups were represented, except for Asians and Pacific Islanders, who were excluded 
because of concerns about the cultural appropriateness of the measures used to assess child and 
family outcomes. Not surprisingly, 96.3 percent of the parents reported that they shared a 
household with their own or their partner’s children.^ Families also had relatively large numbers 
of children, and the majority had preschool-age children living with them. 

Most parents had full-time employment experience, but nearly 20 percent had never 
worked full time. More than half were not working at the time of random assignment, and more 
than one-third had not worked at all in the year prior to random assignment. More than four out 



^The remaining 4 percent of the CFS sample had caretaking responsibilities for children other than their 
own or their partner’s, which could include grandchildren or children of other relatives. 




-25- 



61 



of five were receiving some type of public assistance; the majority (69.5 percent) of those re- 
ceiving assistance were on Aid to Families with Dependent Children (AFDC). More than one- 
third had lived in a public assistance household as a child. 

Many sample members reported some barriers to employment. For example, only 60.6 
percent had graduated from high school or obtained a General Educational Development (GED) 
certificate. More than half reported having no access to a car, while one-fifth reported being ar- 
rested since their sixteenth birthday. 

The Use of New Hope’s Benefits and Services 

Benefit Use by Program Group Members 

New Hope’s designers and staff strove to maximize clients’ participation in the pro- 
gram and their receipt of services. The program was designed to provide a range of services, 
with the expectation that many participants would not need all benefits in every month. Chapter 
1 discusses New Hope’s benefits, and the two-year report outlines the factors that influenced the 
use of benefits by program group members. In order to be eligible for at least some of the bene- 
fits, a sample member had to have worked 30 hours per week in the prior month and household 
income had to be below a specified set of thresholds. 

Benefit take-up was not expected to be universal. New Hope’s health insurance was of 
little interest to participants who were already covered by Medicaid. Community service jobs 
(CSJs) helped only those participants who were not able to secure full-time employment on 
their own. Other reasons for participants’ failure to take up benefits include reluctance to change 
existing arrangements, particularly in the area of child care; lack of awareness of New Hope’s 
flexibility when their circumstances changed, especially in terms of taking advantage of CSJs 
when they lost their jobs; and, for some, fioistration with New Hope’s project representatives 
and confusion about their eligibility. 

The interim report includes information about benefit use during the first two years of 
the evaluation.’ This section presents a complete picture of benefit use through the end of three 
years of eligibility.* All data presented in this section were calculated using data from New 
Hope’s client-tracking database. 



’See Bos et al., 1999. 

*For administrative reasons, a few participants continued receiving benefits through approximately Month 
40 (post-random assignment). For these participants, the data presented in Table 2.2 extend a few months be- 
yond the 36-month eligibility period. 
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Table 2.2 shows the use of earnings supplements, CSJs, health insurance, and child care 
subsidies through the end of eligibility for the sample members who were in the program group. 
The vast majority (87.6 percent) of CFS program group members received at least one of the 
three New Hope financial benefits during their three-year program eligibility period. Almost all 
of these participants (86.5 percent) received at least one earnings supplement. Health insurance 
and child care subsidies were used by slightly more than half the program group members (55.7 
percent and 52.1 percent, respectively). 

The use of earnings supplements and child care benefits over the three years is some- 
what higher than it had been after two years. The use of health care subsidies increased from 40 
percent within two years to 56 percent within three years. It is likely that benefit usage — par- 
ticularly health coverage — increased in the third year because of an increase in the proportion 
of families leaving welfare and, as a result, needing additional supports.® Some evidence of this 
can be found from survey reports for the CFS sample. Although eligibility for Medicaid is offi- 
cially determined independently of an individual’s welfare status, at the two-year follow-up, 
60.9 percent of CFS program group members were receiving Medicaid, compared with only 
22.4 percent at the five-year follow-up. 

Although fewer CFS program group members used health insurance and child care 
benefits than used earnings supplements, all three benefits were used for similar periods of time. 
On average, those who received an earnings supplement did so for a total of 14.9 months, while 
those who used health insurance benefits did so for 12.3 months. Users of the child care subsi- 
dies received such assistance for an average of 15.5 of the 36 months of eligibility. 

The average earnings supplement was $125.89 per month, but supplement amounts 
varied with income and household size. For example, 22.2 percent of CFS families who re- 
ceived a supplement received between $51 and $100 per month, on average. But 14.8 percent 
received an average supplement of more than $200 per month.'® Those who had lower in- 
comes and larger families received the most substantial benefits. Among the households that 
were using New Hope’s HMO health insurance, the average monthly payment was $278 plus an 
additional copayment of $30, which was paid by the participant. The average New Hope contribu- 



®A small part of the increase in receipt of benefits may be due to the fact that this report presents findings 
for a slightly different sample of CFS femilies and that these famihes were slightly more hkely to use benefits 
at the two-year follow-up. Findings for all CFS families in the program group are presented in the interim re- 
port (Bos et al., 1999). This report presents findings for the subset of those families who responded to the five- 
year survey (see the section above entitled “Characteristics of the CFS Sample”). 

'®These numbers do not include the EITC, which was considered in calculating the size of the New Hope 
wage supplement. 
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The New Hope Project 
Table 2.2 

The Use of New Hope’s Financial Benefits and Community Service Jobs 
by Program Group Members, Within the Eligibility Period 



Outcome 


Total 


All households 

Ever used a New Hope financial benefit (%) 


Any type 


87.6 


Monthly earnings supplement 


86.5 


Health insurance 


55.7 


Child care 


52.1 


Ever worked in a CSJ (%) 


32.3 


Average number of months with a financial benefit 
for those who received it 


Any type 


17.7 


Monthly earnings supplement 


14.9 


Health insurance 


12.3 


Child care 


15.5 


Households that received earninss sunnlements 
Number of monthly earnings supplements received (%) 


1-6 


26.6 


7-12 


17.6 


13-18 


18.0 


19-24 


19.7 


25-36 


18.0 


Distribution of amount of monthly earnings supplements (%) 


S1-S50 


17.1 


S51-S100 


22.2 


S101-S150 


23.4 


S151-S200 


17.5 


S201 or more 


14.8 


Average amount of monthly earnings supplement (S) 


125.89 


Households usine New Hope insurance 
Households using each type® (%) 


New Hope HMO health insurance 


70.0 


New Hope contribution toward employer's health insurance 


47.8 


Average New Hope HMO monthly amounts (S) 


Participant contribution 


29.60 


Average monthly benefit (S) 


New Hope health insurance benefit 


277.51 


Total health insurance cost (contribution and benefit) 


307.11 


Average New Hope contribution toward employer's health insurance 


84.52 




(continued) 
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Table 2.2 (continued) 



Outcome 


Total 


Households usins New Hooe child care benefits 




Average monthly amounts ($) 


Participant contribution 


67.05 


New Hope child care benefit 


699.21 


Total child care cost (contribution and benefit) 


766.27 


Sample size 


282 



SOURCE: MDRC calculations using the New Hope Project MIS client-tracking database. 

NOTES: New Hope financial benefits include earnings supplements, child care assistance, and health 
insurance. 

“Some households are in both categories because they may have been part of the New Hope HMO plan 
and then moved to an employer plan (or vice versa). 



tion for employers’ health plans was $85 per month. Child care benefits totaled $766 per month, 
on average — $699 of which was paid by New Hope." 

Figure 2.2 shows that benefit use by program group members remained relatively stable 
in the second half of the 36-month eligibility period, with only a slight decline in the final few 
months of eligibility. At the end of eligibility, 45 percent of the sample were receiving some 
type of New Hope benefit." By Month 40, none of the CFS families were receiving New Hope 
benefits. Box 2. 1 examines some of the reasons for the variation in take-up of the program and 
its benefits. 

Adjusting to the End of New Hope Eligibility 

Since New Hope’s benefits ended after 36 months, there was some concern about how 
families would cope with the ending of their benefits. In the five-year survey, program group 
members were asked about their experience with New Hope benefits and the end of eligibility 
(Table 2.3). Some 59.4 percent of program group members reported that they were receiving 
benefits when their eligibility ended. This proportion is somewhat higher than the proportion 



' 'Average monthly payments for child care were based on only two years of follow-up because additional 
follow-up for this benefit was not available. It is not likely that the average monthly amounts would change 
significantly with additional data. 

'^This is estimated by adding the proportion receiving any benefits in Months 36 through 40. 
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The New Hope Project 
Figure 2.2 

Percentage of Program Group Members Using Any New Hope Financial Benefit 
Within the Eligibility Period, by Month 




Month After Random Assignment 



SOURCE: MDRC calculations using the New Hope Project MIS client-tracking database. 

NOTE: New Hope financial benefits include earnings supplements, child care assistance, and health 
insurance. 
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Box 2.1 



The Take-Up of New Hope’s Benefits Varied Widely 
Because of Family Circumstances 
and Clients’ Understanding of the Program 

Families who were randomly assigned to the program group had diverse experiences 
with the New Hope offer. About half of program group families in the New Hope Ethno- 
graphic Study (NHES) reported really being helped by the program and could explain the 
specific benefits that they received. Another fifth of the NHES families received some benefit 
from the program but were prevented from sustained use of New Hope by other difficulties in 
their lives (such as drug dependency, extreme family crises, and severe mental health prob- 
lems). Nearly another third could have participated in the program but, for various reasons, 
were unable or unwilling to do so in any significant way. 

Drawing on the ethnographic data, Gibson and Weisner examined the various rea- 
sons for the differential rates of New Hope take-up.* Maria, age 41 and a mother of two, de- 
scribed the kind of interest that many parents had and that New Hope both drew on and sup- 
ported: “I was so excited when I got in. I was going to be a participant, and it wasn’t going to 
cost me nothing. If anything, it was going to give me money to just do what I was already do- 
ing.” But Anna, who could work only enough hours to use New Hope’s benefits sporadically, 
despite needing them, had another view: “I had my ups and downs [in New Hope]. . . . Yeah, 
it was still tough, because then you really had to work, because you weren’t getting the hours 
you were supposed to get. You had to make up for it [the 30 hours], and then you still had 
bills and stuff in the way, and things always come up.” 

These two participants illustrate the wide variation in take-up of the New Hope pro- 
gram and benefits. Despite the relative generosity of the program and its supportive case- 
workers, take-up was far from universal, and participants who took up at least some benefits 
rarely took up all of them. Ethnographic analysis of a random sample of all the experimental 
participants found that four categories of personal and family circumstances were associated 
with take-up: (1) the “constrained by information” group (participants whose understandings 
about the program differed from what New Hope offered); (2) the “dismptive life” group 
(who had significant personal troubles and instability); (3) the “pros and cons” group (who 
used explicit cost-benefit calculations to decide about take-up); and (4) the “daily routine” 
group (who took up particular benefits only if they helped sustain the family’s daily routine). 
Analysis of take-up of other services by the control group showed similar patterns, suggesting 
that these take-up patterns existed for other public services and were not specific to New 
Hope. In general, the take-up of antipoverty programs as well as welfare-to-work interven- 
tions like New Hope vary due to the ecological and cultural conditions that families face in 
struggling to sustain a daily routine — including parents’ personal goals and values as well as 
a more conventional cost-benefit approach. 



*Gibson and Weisner, 2002. 
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The New Hope Project 
Table 2.3 

Effects of the End of New Hope Eligibility for the CFS Sample 



Outcome 


Total 


Receiving benefits when eligibility ended (%) 


59.4 


Reported any adjustments due to end of eligibility (%) 


44.2 


Among those who reported adjustments 
Percentage reporting each type“ (%) 


Related to a core New Hope Benefit 


71.8 


Child care 


38.0 


Health insurance 


22.3 


Other 


33.4 


Employment-related 


26.1 


Practical coping adjustment 


29.8 


Negative consequences 


0.8 


Other 


12.3 


Among those who were receiving benefits when eligibility ended 
Percentage who reported that at the end of New Hope they were: (%) 


Completely ready to move forward 


37.5 


Somewhat ready to move forward 


44.4 


Not at all ready to move forward 


18.1 


Percentage who reported that the end of eligibility was; (%) 


Not a problem at all 


43.4 


A minor problem 


37.7 


A major problem 


18.9 


Sample size 


271 



SOURCE: New Hope five-year survey. 

NOTE: “Respondents may have reported more than one adjustment type; therefore, categories do not sum to 
1 00 percent. 
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reported earlier, which was calculated using the New Hope client-tracking database, and it sug- 
gests some degree of recall error in responses to the end-of-eligibility questions. 

Just under half (44.2 percent) of all program group members and three-quarters of fami- 
lies still receiving benefits reported that they had to make some type of adjustment in response 
to the termination of their benefits. Most of the adjustments concerned one of the core New 
Hope benefits: earnings supplements, child care, or health insurance. Other common adjust- 
ments were related to employment (for example, the 26.1 percent reporting employment-related 
adjustments) and to practical coping adjustments such as budgeting, changing work and educa- 
tion schedules, and changing child care arrangements (29.8 percent). 

New Hope’s designers and staff were concerned about the loss of benefits, and they 
worked to ensure that participants understood that benefits would end and to ensure that the ad- 
justment to the loss of benefits would go as smoothly as possible. Perhaps as a result, most par- 
ticipants reported no problems. Eighty-two percent of those who were still receiving benefits 
when their eligibility ended reported being somewhat or completely ready to move forward 
without New Hope, and 43.4 percent reported that the end of eligibility was not a probleni at all. 
However, about one-fifth of the families reported that they were not ready to move forward 
without New Hope and that the end of eligibility was a major problem. Box 2.2 gives examples 
of how two families adjusted to the end of New Hope’s benefits. 



Comparison of Benefit Use by Program and Control Group 
Members 

The two-year survey provided some evidence that New Hope helped families learn 
about and use other supports that were available in their communities. The five-year survey 
permits a test of whether this was the case at least two years beyond the end of the New Hope 
program. The upper panel of Table 2.4 compares health and child care benefit receipt at 24 
months for program and control group members, and the lower panel of the table presents simi- 
lar outcomes at the five-year follow-up. 

Two points must be kept in mind when reading Table 2.4. First, the program effects 
shown are based on a different data source and for a different time period than findings shown 
earlier in the chapter. The earlier findings are based on three-year administrative data fi’om the 
New Hope program, but such data do not capture the use of non-New Hope services by pro- 
gram group members or the use of comparable services by individuals who were randomly as- 
signed to the control group. Table 2.4 uses data fi’om the two follow-up surveys, which were 
administered to both New Hope participants and control group members. 

Table 2.4 is also the first table in the report that shows experimental program effects (or 
“impacts”). These effects are based on mean differences between the program and the control 
group families that have been adjusted for the mostly small differences in their characteristics at 
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Box 2.2 



The Loss of New Hope’s Supports Affected Families 

Very Differently 

The families in the New Hope Ethnographic Study (NHES) and those in the five- 
year survey sample reported similar experiences in leaving the program. Of course, many 
New Hope families had been receiving program support only sporadically and at the end 
of their period of eligibility were not receiving benefits at all. For these families, the im- 
pact of leaving was minimal. The NHES families who seem to have been most impacted 
by the end of their period of eligibility had been using the program’s benefits more or less 
continuously over the entire period of eligibility. These families faced significant cost in- 
creases to secure services like those they were receiving through New Hope — particu- 
larly families who were long-term users of the program’s health insurance and child care 
benefits. Many of these families also lost access to services that they felt were particularly 
beneficial to their children, such as a child care provider. While some of these families 
were able — within a few months of leaving the program — to find alternatives that re- 
placed the services that New Hope had paid for, other families stmggled with the loss of 
program supports for years. 

L’Kesha, an Afiican-American single mother of two school-aged children and 
one preschool-aged child, reported on the survey that the end of New Hope eligibility was 
“no problem at all.” Although she had been receiving the wage supplements, she mainly 
used them to buy “extras” for herself and the children, so she said she did not really miss 
them. The child care supplement was another story, however. L’Kesha worked full time 
as a receptionist for a local nonprofit organization, and she used the child care supplement 
to pay a woman who reliably cared for her children during L’Kesha’s entire term of en- 
rollment. Once the family’s eligibility ended, the cost of this arrangement nearly quadru- 
pled (fi'om about S25 per week to nearly S90 per week). As a result, L’Kesha could no 
longer afford child care with this provider. For a period of several months after leaving 
New Hope, L’Kesha relied on a number of informal child care options involving her rela- 
tives and her boyfiiend. She also tried to bring her youngest daughter to work with her a 
few times, but her supervisors disapproved of that. By the fall of 1998, with her youngest 
child starting kindergarten, L’Kesha relied on her boyfiiend to help get the children to 
school in the morning, and her niece would watch them after school. Although this ar- 
rangement was not as good as the arrangement with the earlier child care provider, it 
proved stable over time. 

Edith’s story is similar to L’Kesha’s, but she experienced her situation as being 
much more difficult. She reported on the survey that the end of New Hope eligibility was 
“a major problem.” Edith, a married Latina mother whose three children were between 
the ages of 2 and 6 in 1998, also mainly relied on New Hope’s child care supplement and 
with it was able to keep all her children in a formal daycare center that she believed was 
of very high quality: “There the children are sons of doctors, lawyers, policemen, and I 
wanted my children to be among those with [parents] of a higher social status.” The day- 
care was very good for her youngest daughter, Libertad, because it promoted better mas- 
tery of language at an early age, in Edith’s view. The daycare also provided educational 
games to play that aided the children’s development. However, once the family’s eligibil- 
ity for New Hope came to an end, Edith’s cost to keep her children in this care arrange- 
ment increased from about SI 00 a month to over S800 a month. She applied for Wiscon- 
sin Works (W-2) child care, but her household income exceeded the thresholds for eligi- 
bility. On a friend’s recommendation, Edith resorted to leaving her kids with a woman 
who ran a babysitting service in her home. Edith was unhappy with the quality of this ar- 
rangement, however, often complaining that the woman put the children in front of the 
television all day to keep them quiet. But this kind of care was all that Edith could afford. 
Over the next few years, she tried several other babysitters but never found the quality of 
care that her children had received when the family was enrolled in New Hope. 
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The New Hope Project 
Table 2.4 



Impacts on the Use of Health and Child Care Benefits at Two Years and at Five Years 





Program 


Control 




P-Value for 


% 


Effect 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size® 


Two-vear impacts** 














Currently receiving health insurance‘s (%) 


93.5 


85.5 


8.0 *** 


0.002 


9.3 


0.20 


Type of health insurance (%) 














New Hope plan 


37.4 


n/a 


n/a 


n/a 


n/a 


n/a 


Medicaid'* 


60.9 


67.4 


-6.5 * 


0.077 


-9.6 


-0.13 


Employer plan 


38.3 


37.9 


0.4 


0.918 


1.1 


0.01 


Receiving child care assistance, 
any type (%) 


59.4 


41.0 


18.5 *** 


0.000 


45.1 


0.39 


New Hope subsidy 


50.0 


n/a 


n/a 


n/a 


n/a 


n/a 


Welfare department subsidy 


15.6 


41.4 


-25.8 


0.000 


-62.4 


-0.61 


Other subsidy 


1.2 


0.3 


0.8 


0.341 


240.4 


0.10 


Sample size 


289 


301 










Five-year impacts'* 














Currently receiving health insurance (%) 














Adult covered by health insurance 
All focal children covered by 


85.9 


88.1 


-2.2 


0.435 


-2.5 


-0.07 


health insurance 


84.2 


88.2 


-3.9 


0.179 


-4.5 


-0.11 


Type of health insurance^ (%) 














Employer plan 


33.2 


31.9 


1.3 


0.739 


4.1 


0.03 


Medicaid** 


22.4 


27.6 


-5.3 


0.143 


-19.1 


-0.12 


BadgerCare*^ 


20.1 


21.0 


-0.9 


0.799 


-4.2 


-0.02 


Other private or family employer plan 


14.3 


13.6 


0.8 


0.796 


5.6 


0.02 


Receiving child care subsidy from 














welfare or other agency (%) 


16.4 


16.7 


-0.3 


0.913 


-2.0 


-0.01 


Sample size 


282 


279 











(continued) 
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Table 2.4 (continued) 

SOURCES: New Hope two- and five-year surveys. 

NOTES: *The effect size is the difference between program and control group outcomes expressed as a 
proportion of the standard deviation of the outcome for both groups combined. This standard deviation is always 
obtained from the entire research sample, even if the table shows impacts for subgroups. 

^Some outcomes presented in this table may differ across the two follow-up points. Because of differences 
in the New Hope program status, the survey questions asked at the two-year follow-up were slightly different 
from those asked at the five-year follow-up. 

Categories of health insurance are not mutually exclusive. Because a person may have received more than 
one type of insurance, the subcategory percentages need not sum to the total percentage covered by health 
insurance. 

‘^Includes coverage of spouse/partner and children. 

A state-sponsored program in Wisconsin that offers Medicaid insurance to low-income, uninsured families 
with children. 



baseline. Box 2.3 explains how to interpret such a table; from this point on, all tables that show 
program effects will follow this general format. 

The upper panel of Table 2.4 shows that, at the two-year point. New Hope substantially 
increased the likelihood of families’ being covered by health insurance and receiving child care 
subsidies. The lower panel of the table shows that these effects did not persist; at the time of the 
five-year survey, there were no significant program effects on benefits received. Although most 
families were covered by health insurance, neither adults nor the children in New Hope families 
were any more likely than their control group counterparts to have health insurance at the five- 
year follow-up. Between 85 percent and 88 percent of both program and control group members 
reported having access to health insurance. Nor did New Hope’s impact on the proportion of 
families receiving any type of child care subsidy persist through five years: About 16 percent of 
both groups reported receiving child care assistance. 



The Cost of Providing New Hope’s Services 

The two-year follow-up report presented an early look at the costs and benefits of the 
New Hope program. This section expands on that analysis by presenting costs for the CFS sam- 
ple and by extending the cost estimates for an additional year — through the end of eligibility 
for New Hope. 
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Box 2.3 



How to Read an Impact Table 

Table 2.4 is the first in a series of tables featuring program-control group differ- 
ences for separate subgroups in New Hope. These differences constitute estimates of pro- 
gram effects and are also referred to as “impacts” throughout the report. In addition to es- 
timates of program effects, each table contains a great deal of information about the sig- 
nificance of the effects, how to interpret their size, and how they compare with other pro- 
gram effects. Not all of this information is of use to all readers, but all should be able to 
fmd what they need in these tables. 

Here is a column-by-column description of the features in a typical impact table 
in this report; 

1. The first three columns of data (“Program Group,” “Control Group,” and “Differ- 
ence”) tell the basic impact story. They show the outcome levels for program and 
control group members and the difference between these levels, which is the estimate 
of the program effect. The unit in which these impacts are expressed is shown at the 
left, in the table stub; the unit is usually either a percentage or a dollar amount, but 
other units are used as well. 

2. Statistical tests are conducted to assess whether the differences shown in the third 
column are statistically significant (using a two-tailed significance test). Asterisks 
next to the differences indicate the results of these tests. The absence of an asterisk 
means that the probability is 10 percent or greater that the difference is only the result 
of random chance. Three asterisks indicate the highest level of statistical significance; 
the chance of a difference that is really zero is less than 1 percent. Two asterisks 
mean that this chance is between 1 percent and 5 percent, and one asterisk means that 
it is between 5 percent and 10 percent. 

3. The fourth column shows “P-Value for Difference.” This is the exact probability that 
the impact is really the result of random chance. It is useful to refer to the p- value for 
impacts that are marginally statistically significant, which often happens when sam- 
ple sizes are small. For example, the impact on Medicaid receipt shown in the lower 
panel of Table 2.4 was not statistically significant, but the probability of an entirely 
random difference was only 0. 143. 

4. The fifth column shows “% [Percentage] Impact.” This is the difference expressed as 
a percentage of the control mean. These numbers are useful to compare impacts that 
are based on different units (such as dollars and hours of work) and to compare im- 
pacts across outcomes with very different means (like monthly and quarterly earn- 
ings). 

5. The sixth column shows “Effect Size.” This is the impact divided by the foil sample’s 
standard deviation for the outcome. Effect sizes are widely used to compare efects 
across different programs and across different outcome areas. Researchers like to 
make statements about the “absolute” size of effects based on these effect sizes. As a 
rule of thumb, effect sizes of 0.1, 0.3, and 0.5 are considered small, medium, and 
large, respectively. However, generally it is beUeved that effect sizes for mediating 
outcomes (for example, participation, attitudes, goals) need to be larger to be mean- 
ingful than effect sizes for “final” outcomes (for example, income, graduation, school 
progress).* 



*For a detailed (and classic) discussion of effect sizes, see Cohen (1988, pp. 53 1-553). 



Before beginning the discussion of program costs, it is important to understand the limi- 
tations of this analysis. Unlike the two-year report, this section presents only the gross costs of 
providing New Hope’s services to CFS program group members.'^ Thus, in this analysis, any 
savings in government expenditures as a result of New Hope are not subtracted from the costs 
of ruiming the program. Such savings might include decreased welfare expenditures, Medicaid, 
and child care subsidies. The analysis that is presented in the two-year report estimated that 
these savings totaled $944. Because of New Hope’s impacts on employment, the government 
budget might also benefit from increased revenue brought on by increased income and sales 
taxes, which were estimated in the two-year report to total $96 at the end of two years. Nor does 
this analysis account for any benefits to program group members as a result of New Hope. 

The costs presented in this section are not directly comparable to the costs presented in 
the earlier report because they are estimated for the CFS sample only, whereas the two-year re- 
port presents costs for the entire program group.''* Analyses of participation patterns at the two- 
year follow-up showed that CFS families’ participation rates did differ from non-CFS families, 
particularly in the area of New Hope child care subsidies and health insurance. Program costs 
for CFS families are higher on average, largely because child care costs are such a large com- 
ponent of program costs and because CFS families had more children in child care during the 
follow-up period.'^ 

Table 2.5 presents estimates of the gross costs of providing New Hope services to CFS 
program group members for the three-year eligibility period.'* The total cost of New Hope for 
CFS families was $15,799, or approximately $5,270 per year per CFS program group family. 
The table shows that the largest percentage of program costs came from child care subsidies 
(37.9 percent), followed by case management and administration of benefits (22.8 percent). 



Conclusion 

The participation figures presented in this chapter clearly show that New Hope did not 
provide the full range of program benefits to every participant in every month. However, the 



'^A more complete accounting of costs and benefits will be presented in a later report. 

'‘’Although the costs in this section account for variations in the use of program services for CFS families 
compared with non-CFS families and program group members as a whole, the participation rates that were 
used are for the 24-month follow-up period, due to data constraints. An analysis of available data shows that 
third-year participation rates were lower for child care subsidies and higher for New Hope health insurance. 

‘*It is very likely that some of the increased costs for child care were offset by savings in other types of 
child care assistance offered by the welfare department and other government agencies. 

'*To calculate the estimates of costs presented here, the unit costs used in the two-year report were applied 
to an additional year of eligibility. For a more detailed discussion of the calculations and the underlying as- 
sumptions of the analysis, see Bos et al. (1999). 
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The New Hope Project 
Table 2.5 



Estimated Three-Year Gross Costs of New Hope per CFS 
Program Group Member (in 1996 Dollars) 



Gross Cost ($) 


Program Group 
Member 


Percentage 
of Total 


Pro 2 ram comoonent 






Earnings supplement 


1,373 


8.7 


Health benefits 


1,849 


11.7 


Child care subsidy 


5,987 


37.9 


Community service job (CSJ) wages® 


1,443 


9.1 


Program administration*^ 


1,545 


9.8 


Case management, benefit administration, 
development and management of CSJs 


3,604 


22.8 


T otal gross program cost for three years 


15,799 




Sample size 


366 





SOURCE; MDRC calculations using expenditure data from the New Hope Project accounting system. 



NOTES; “The $4.75 hourly minimum wage became effective on October 1, 1996. Prior to that date, it was 
$4.25 per hour. On September 1 , 1 997, it became $5.15 per hour. 

’’Includes overhead expenditures such as rent, utilities, and depreciation. 



vast majority of CFS program group members (87.6 percent) received at least one New Hope 
benefit at some point during the follow-up. In an average month, 38.7 percent of program group 
members received some type of New Hope benefit. Specifically, in an average month, 35.0 per- 
cent received an earnings supplement; 18.6 percent were covered by New Hope health insur- 
ance; and 22.3 percent used New Hope child care assistance. Another 6.5 percent worked in a 
CSJ. Families who received New Hope benefits tended to receive them for a fair number of 
months — a year or more, on average. 

There is some evidence that families who did not originally use any New Hope bene- 
fits began receiving earnings supplements and child care subsidies later in the follow-up pe- 
riod. Most notable is the proportion who began to participate in the New Hope health plan in 
the last year of eligibility, perhaps as access to other sources of insurance decreased. The in- 
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crease in the number of families using benefits demonstrates that even though New Hope did 
not consistently serve every family in every month, it did offer a range of work supports that 
were used by many families. 



Differences in benefit receipt that were noted earlier in the two-year follow-up period 
did not persist two years beyond the end of New Hope’s benefit period. Five years after base- 
line, equal numbers of program and control group families were covered by health insurance 
and were receiving child care subsidies. Although a significant number of families were receiv- 
ing New Hope benefits when their eligibility ended, the majority reported no hardships associ- 
ated with the loss of benefits. 




ERIC 



Chapter 3 



New Hope’s Effects on Employment, 
Benefit Receipt, and income 



This chapter presents New Hope’s effects on employment, earnings, receipt of public 
assistance, and income over the five-year period following random assi gnm ent Program effects 
on these outcomes are presented first for the entire Child and Family Study (CFS) sample and 
then for important subgroups.' 



Key Findings 

• New Hope increased employment over the five-year period, with the bulk of 
the increase occurring within the first two years. Over the 60-month period 
following random assignment, program group members worked more total 
quarters and more consecutive quarters than did control group members. 

• New Hope increased eamings-related income and total income over the five- 
year period, although the size of the effects diminished after Year 3, when 
the program supports ended.^ Even though the impacts on total income were 
not large during the last two years, poverty rates were significantly lower 
during this period because New Hope reduced the number of families with 
very low incomes. 

• New Hope had no significant effects on benefit receipt fi'om the Aid to Fami- 
lies with Dependent Children (AFDC) program or the Temporary Assistance 
for Needy Families (TANF) program. 

• New Hope increased long-term stable employment. This increase may have 
contributed to the fact that the program group also earned somewhat higher 
wages than the control group in Year 5. 

• New Hope had differential effects on subgroups. For instance. New Hope 
had lasting effects on those sample members with one of a number of possi- 
ble barriers to employment, and it helped Hispanics more than it helped 
blacks (non-Hispanic) or whites (non-Hispanic). 



'An analysis for the total New Hope sample of 1,357 members — including adults who are not part of the 
CFS sample — will appear in a separate report. 

^Recall that funding constraints limited the duration of New Hope’s benefits to three years. 
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New Hope’s Rationale and Theoretical Framework 

The New Hope program was designed in response to serious shortcomings in the em- 
ployment situation of many low-income residents of Milwaukee. Despite substantial job growth, 
low unemployment, and a very favorable economic situation, many low-income workers hold 
part-time jobs when they want to work lull time, are intermittently unemployed, or earn a wage 
that is not sufficient to lift their families out of poverty. In addition, most low-wage jobs do not 
offer health benefits, even though most Americans consider such benefits a necessity. Also, many 
low-income families must spend substantial amounts of money to place their children in child care 
while they work. The lack of subsidized child care is well documented and is widely considered to 
be a substantial barrier to the labor force participation of low-income parents.^ 

New Hope’s Response 

To address these problems in the context of a demonstration program. New Hope of- 
fered its participants an array of services and benefits. These services and their potential effects 
on employment, earnings, public assistance, and income are summarized in Figure 3.1, which 
expands the left-hand portion of the conceptual model presented in Chapter 1 (Figure 1.1). First, 
New Hope developed community service jobs (CSJs), which were available to any participant 
who was willing to work full time (30 hours a week or more) but was unable to find full-time 
employment."' CSJs were designed to be temporary, allowing participants to bridge spells of 
unemployment or underemployment, to become acquainted with the world of work, or to find 
new career paths when previous jobs had not worked out. 

A key New Hope benefit was its earnings supplement, which was available to anyone 
working at least 30 hours a week. The supplement “topped up” the net earnings of full-time 
workers to the federal poverty standard (after accounting for the federal and state Earned In- 
come Tax Credits [EITCs]), in order to eliminate the “poverty-wage” employment that is so 
common among low-wage workers. 

Outcomes for New Hope’s control group provide an indication of the need for the 
earnings supplement component in Milwaukee’s low-wage labor market. As shown in the 
two-year report, for a majority of people in the control group, earnings combined with EITC 
benefits in Years 1 and 2 failed to bring family incomes above the poverty line. Although 
poverty is less pervasive than it would have been in the absence of the EITC, there still ap- 
pears to be a substantial need for income support to low-income workers. These conditions 



^Fuller, Kagan, Caspary, and Gauthier, 2002. 

“'For more details about the community service jobs, see Poglinco, Brash, and Granger (1998). 
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Conceptual Model of New Hope's Impacts on Earnings, Income, and Public Assistance 
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remained two years after sample members first applied to New Hope, and they occurred de- 
spite Milwaukee’s tight labor market. 

Health insurance and access to health care are ongoing and growing concerns for low- 
income families. At the two-year follow-up of New Hope, less than half of control group mem- 
bers who were employed had an employer who offered health care benefits. Of the remainder, 
many were eligible for Medicaid, either because they received some public assistance benefits 
or because they met the income cutoff for Wisconsin’s BadgerCare program.^ However, as 
shown in the two-year report, nearly 60 percent of the control group sample reported experienc- 
ing periods without health insurance during the first two years of follow-up. 

Lastly, New Hope attempted to address low-wage workers’ need for affordable child 
care. There is a growing literature documenting the difficulties that low-wage workers — espe- 
cially women — face in finding and paying for good-quality child care while they are at their 
jobs.* More and more jobs have irregular or unconventional hours, when schools and formal 
child care centers are closed. And even if adequate child care is available, it usually involves 
long waiting lists and substantial fees. 

Thus, there is a range of potential supports that low-income workers need in order to 
stay employed and bring their families out of poverty. Chapter 2 shows that most New Hope 
participants availed themselves of one or more of the program’s services, and the two-year 
evaluation indicates that most participants expressed very favorable opinions about the program 
and its staff. Given all that, why does this chapter rely on comparisons of participants and con- 
trol group members to assess the effects of New Hope on employment, earnings, and income? 
The answer to this question lies in the expectation that New Hope’s services would affect the 
behavior of program participants and the choices they make. 

Effects on Job Decisions 

Changes in behavior caused by New Hope could either magnify or reduce the pro- 
gram’s benefits for its participants. Consider the theoretical model in Figure 3.1. The left side of 
this model lists all the components of New Hope that are discussed in Chapter 2. Arrows con- 
nect these components to the key outcomes discussed in this chapter. The figure does not pre- 
sume to be a comprehensive depiction of reality: The arrows link only to the outcomes that the 
program was most likely to affect directly, and not all possible outcomes are shown. 



^BadgerCare is a state program offering free medical insurance to families whose income is below a cer- 
tain cutoff level but not low enough to quahfy them for pubhc assistance benefits. 

^Fuller, Kagan, Caspary, and Gauthier, 2002. 
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In the second column of Figure 3.1, two areas of employment outcomes that might be 
affected by New Hope are distinguished. First, there is the program’s direct effect on employ- 
ment decisions. By providing earnings supplements and assistance with child care and health 
care. New Hope increased the immediate payoff from work while simultaneously reducing 
some of the costs associated with going to work. Coupled with a 30-hour minimum for the av- 
erage weekly work effort to qualify for benefits, these factors might be expected to increase (1) 
the number of participants who seek employment and (2) the number of hours that they work 
(their work effort, shown in the model’s third column).’ This expected response is one of the 
key assumptions underlying the concept of “making work pay.” 

Second, New Hope provided community service jobs. CSJs were available to those 
who could not find employment on their own, despite an eight-week job search. In addition. 
New Hope project representatives actively assisted and supported participants in their search for 
regular employment. Consequently, one might expect the access to jobs and the range of job 
options to be increased by New Hope, as shown in the second column of the model. This, in 
turn, would affect the employment decisions and actual employment outcomes of those partici- 
pating in the program. Again, the initial expectation was that New Hope would increase em- 
ployment as a result. 

The third column of Figure 3.1 shows how New Hope was expected to increase partici- 
pants’ incomes — both directly, by providing financial and in-kind assistance, and indirectly, by 
increasing the earnings from work. Such increases in income (and financial stability) can affect 
subsequent choices that people make. Specifically, economic literature and past experience with 
programs that transfer income predict that those who benefit from such transfers may decide to 
reduce their work effort in favor of child care, family time, social activities, or leisure.* This can 
produce a negative effect on employment that, for some participants, could offset the positive 

’The effect of the 30-hour rule on participants’ response to the program is not altogether obvious. While 
this threshold substantially increases the incentive for those not working at all and for those working substan- 
tially fewer than 30 hours, it also introduces a “hurdle” that may be difficult to overcome for some participants 
— especially those with limited work experience or substantial barriers to employment. If, for some reason, a 
person simply carmot work 30 hours per week, the program’s benefits (and thereby its potential effect on be- 
havior and subsequent outcomes) are severely curtailed. 

*There is an extensive economic hterature on these responses to changes in income from transfers and subsi- 
dies. Much of this literature discusses a series of social experiments conducted in the early 1970s, known as the 
Negative Income Tax (NIT) experiments. In these experiments, it was found that increasing the income of poor 
families by using tax-based subsidies reduced the labor supply (work effort) of the affected famihes, but not uni- 
versally so and usually not to any substantial degree. For a discussion of the underlying theory and findings from 
these NIT studies, see, for example, Robins, Keeley, Spiegelman, and West (1978); Robins, Spiegelman, Weiner, 
and Bell (1980); Danziger, Haveman, and Plotnick (1981); Ashenfelter (1978); Ashenfelter and Plant (1990); 
Killingsworth (1976); and Robins and West (1983). For more recent examples, see Card and Robins (1996) and 
Berlin et al. (1998). For a discussion of the role played by marginal taxes and labor supply in the design of the 
New Hope offer, please refer to Brock, Doohttle, Fellerath, and Wiseman (1997, pp. 200-2 1 3). 
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effects mentioned above. Specifically, one might expect such negative effects to occur for those 
who already work well over 30 hours a week and do not need the program’s help in finding a 
job. Especially among the working poor, who already often must hold several jobs to break 
even, the added income fi-om a program like New Hope might offer a good opportunity to settle 
into a more manageable work life. The expected effect in that case would be a reduction in 
work effort. 

Finally, the model acknowledges the possibility that the program’s effects on employ- 
ment and earnings could translate into reduced public assistance for its participants. That is, 
both because of welfare rules and because of individual choice, participants may receive fewer 
public assistance payments than they would have in the absence of the program. While this 
dampens the program’s potential effects on sample members’ incomes, it also generates pro- 
gram benefits for the government and for taxpayers. 

Measuring Program Impacts 

When implementing a set of benefits such as those offered by the New Hope program, 
it is important to document all these behavioral responses. Even when the use of actual program 
benefits is well documented, the real “impact” of the program includes both these benefits and 
the behavioral responses they elicit. For example, a New Hope participant may have been much 
better off because her income was increased with an earnings supplement and her child care 
needs were heavily subsidized by New Hope. But if such help fi'om the program also caused her 
to leave welfare for work, the overall benefit was much greater than the amount of money trans- 
ferred through New Hope. 

It is difficult to assess behavioral changes resulting fi-om participation in New Hope by 
looking only at the participants. As their lives changed over time, some of this change might 
have been attributable to their contact with New Hope while other changes might have hap- 
pened anyway. Therefore, an independent assessment must be made of what would have hap- 
pened to New Hope participants in the program’s absence. In this case, such an assessment is 
based on the experiences of the control group. 



Data Sources 

New Hope’s impacts on employment and income were assessed primarily by using data 
fi-om administrative records. Earnings and employment outcomes were measured with unem- 
ployment insurance (UI) earnings records, which were available for all 745 Child and Family 
Study (CFS) sample members in the evaluation. Because these data are limited to quarterly 
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earnings reports, they do not cover many interesting details about sample members’ employ- 
ment experiences.’ Therefore, for impacts on hours of work, hourly wage rates, or month-to- 
month changes in employment status or the type of job held, the report relies on survey data. 
The five-year follow-up survey was completed by 561 survey sample members (75 percent of 
the CFS sample). In what follows, impacts on earnings, welfare, and income are estimated for 
the entire CFS sample of 745 families, while impacts on job characteristics are estimated for the 
survey sample of 561 families. To assess the extent to which the findings vary depending on the 
data sample used, an analysis of response bias was conducted. The results fi'om this comparison 
are presented in Appendix C and show a few differences between the impacts for the CFS sam- 
ple and the five-year survey sample. However, these discrepancies do not appear to have had 
significant consequences for the estimates of New Hope’s program effects. 

Earnings fi'om employment were not the only income source available to New Hope 
sample members. Therefore, this evaluation obtained follow-up data fi'om several other adminis- 
trative sources, including the Wisconsin tax system (for information about the use of federal and 
state EITCs), the AFDC system (now replaced with Wisconsin Works, or W-2), food stamp data- 
bases, and the Medicaid system. For New Hope participants, these data are supplemented with 
various measures of program benefits, such as the earnings supplement, earnings fi'om CSJs, and 
health and child care benefits — all collected fi'om New Hope program administrators. 

A key advantage of administrative data sources is that they enable the construction of 
longitudinal measures of employment, earnings, and public assistance receipt, showing how 
participants fared over time. However, a disadvantage is that administrative records do not 
cover all possible sources of household income. Most of these data are available only for one 
person in each household. This person, the “primary sample member,” provided his or her So- 
cial Security number and other identifying information to New Hope at the time of application 
to the program.'® Although it is known that other household members often worked and con- 
tributed income to the primary sample members’ households, there is no way of knowing ex- 
actly how much they worked and how much income they contributed to the household. Also, 
many income sources are not captured by the administrative data but may be very important to 
some households in the New Hope sample. Examples of such income sources include General 
Assistance, Supplemental Security Income (SSI), alimony or child support, child care subsidies 
outside New Hope, and financial help fi'om family and fiiends. Thus, it is likely that continuous 

’Administrative data also fail to capture earnings for sample members who move or work out of state and 
for those who are self-employed. 

'®A relatively small number of sample members provided a second Social Security number for another 
working member of the household. Information for this second person is included in the analysis, but it is not 
appropriate to aggregate income from both household members into a single measure. This is the case because 
Social Security numbers were not obtained for new household members and because it is unknown whether 
original household members left the household. 
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measures of household income underestimated the amount of income actually available to New 
Hope sample members. For individual families, such underestimates could be quite large. For 
example, Box 3.1 discusses how child care impacts may have occurred due to New Hope. 
However, this source of income is not included in administrative records and in the impacts pre- 
sented in this chapter. Chapter 4 uses data from the survey to measure the program’s impacts on 
total household income from all sources. 



New Hope’s Effects 

This section presents New Hope’s effects on employment, benefit receipt, and income. 
Before presenting the detailed results. Figures 3.2 through 3.4 summarize the program’s effects 
over time on three key outcomes: employment, earnings, and total income. The leftmost area of 
Figure 3.2 shows that the program group looked very similar to the control group, in terms of 
employment, prior to entering the New Hope program." The large difference in employment 
between the groups after random assignment shows that New Hope made employment more 
attractive and enabled program group members to find work. Employment for those in the pro- 
gram was fairly stable over the period, falling slightly, from 75 percent in Quarter 1 after ran- 
dom assignment to about 72 percent by Quarter 20. New Hope’s effects on employment were 
largest in Quarters 1 through 6 and diminished thereafter. By Quarter 14, employment rates 
were similar for both groups. The effects on earnings show a similar pattern, with the earnings 
impacts being largest in the early quarters (Figure 3.3). However, earnings were still somewhat 
higher for the program group in the last two years of follow-up, even though the differences 
were not statistically significant. 

Finally, Figure 3.4 presents New Hope’s effects on total income in each follow-up year. 
Total income is the sum of earnings, EITC benefits. New Hope supplements, welfare benefits, 
and food stamps. Consistent with the impacts on earnings, the program’s effects on total income 
were largest in Years 1 through 3, during which New Hope was still operating and providing 
earnings supplements. The program group continued to have higher income than the control 
group in Years 4 and 5, although the differences were not significant. As the subsequent tables 
show, the higher total income for the program group in the later years was due to higher earn- 
ings and EITC benefits, since the program had little effect on benefit receipt in Years 4 and 5. 



"Quarterly employment rates for the control 
absence of New Hope. 



group (Figure 3.2) show what would have happened in the 



Box 3.1 



New Hope Lessened Strains on Informal Social Relationships 

Ethnographic studies of low-income families have clearly documented the signifi- 
cance of informal social ties (for example, family and friends) in helping make ends meet and 
in sustaining critical daily productive and reproductive activities.* For many families, it 
would simply be impossible to make financial ends meet or successfully accomplish many 
daily tasks without the support of informal social relationships. While these relationships gen- 
erally offer support that costs less in financial terms than similar services might cost in the 
market, they do have other “costs” and must be carefully managed if they are to be reliable 
over time. The financial supports and benefits offered through New Hope helped many fami- 
lies better manage their resources and relieve some of the strain that might otherwise have 
burdened their informal sources of social support. The cases of Samantha and Alicia provide 
contrasting examples. 

Samantha, an African-American single mother of four young children, was a member 
of the control group. When asked how New Hope might have helped had she been selected, 
Samantha described how she could have really used the child care and the wage supplement 
to help her sustain full-time employment and better make ends meet. In the absence of these 
supports, Samantha often relied on members of her social networks to help her with child care 
and emergency financial needs. For example, while working full time at a local video store, 
Samantha relied on her mother to watch her children. But this arrangement never seemed se- 
cure. Samantha worried that her mother “would go nuts” if she cared for all four children for 
too long a period of time. As a result, she was anxious to find another arrangement once the 
school year ended and her two oldest children would be home all day for the summer recess. 

Moreover, when faced with occasional financial crises, Samantha felt that she could 
only really rely on a woman from her church. For example, when pregnant with her youngest 
child and in need of a new washing machine, Samantha borrowed about $300 from this 
woman. Samantha was very careful to repay this woman, at the rate of $50 a month, until the 
debt was paid off. She did not want “to mess up” with this friend. If she ever had another fi- 
nancial crisis, she wanted to be able to rely on this friend again. 

On the other hand, Alicia, a married Latina mother of one and a member of the pro- 
gram group, described how support from New Hope helped her to not overburden her infor- 
mal social support network. Alicia mainly used the health insurance benefit from New Hope, 
with only occasional use of the wage supplement and child care. The health insurance was a 
great benefit to her. She was able to save extra money from her job that would otherwise have 
been spent on health insurance, and the extra money afforded her a greater degree of comfort 
in meeting her financial needs from month to month. New Hope also helped Alicia to better 
negotiate community resources. She found English language classes, workshops on employ- 
ment and asset development, and occasional child care for her daughter particularly helpful. 
Alicia found a great deal of support from these community services, and she preferred this to 
relying on members of her informal social network for support. Alicia believed that her in- 
formal social supports were stronger (because they were not overused, with the accompanying 
expectations on her as well) than they would have been without New Hope’s support. 

Samantha and Ahcia described family circumstances in which New Hope helped (or 
could have helped, in Samantha’s case) indirectly, through ripple effects on other aspects of 
their work lives, household economy, and social support networks. This was a common kind of 
benefit of New Hope’s strategy of offering a suite of benefits from which famihes could choose. 



♦Stack, 1974; Edin and Lein, 1997; Newman, 2000. 
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Employment (%) 



The New Hope Project 
Figure 3.2 

Quarterly Employment, by Research Group 




Quarter After Random Assignment 

SOURCES: MDRC calculations using data fi*om the New Hope Project MIS client-tracking database and 
Wisconsin unemployment insurance (UI) records. 
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Earnings ($) 



The New Hope Project 
Figure 3.3 

Quarterly Earnings, by Research Group 




Quarter After Random Assignment 

SOURCES: MDRC calculations using data from the New Hope Project MIS client-tracking database and 
Wisconsin unemployment insurance (UI) records. 
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The New Hope Project 
Figure 3.4 

Total Income Over the Five-Year Period, by Research Group 




B 

o 



o 

H 




Year 



SOURCES: MDRC calculations using data from the New Hope Project MIS client-tracking database, Wisconsin 
unemployment insurance (UI) records, and Wisconsin Department of Workforce Development, AFDC, and Food Stamp 
Program records. 



NOTE: Total income is calculated as the sum of earnings, EITC benefits. New Hope sqDplements, welfare benefits, and 
food stamps. 



Effects on Employment and Earnings 

Table 3.1 presents New Hope’s effects on summary measures of employment, earnings, 
and eamings-related income.'^ The table presents the annual-average of employment and earn- 
ings during Years 1 to 5 and the average over the entire five-year period. 

New Hope significantly increased the proportion of individuals employed in the first 
two years of the program. Employment among New Hope participants was 8.2 percent and 7.3 
percent higher than control group employment in Year 1 and Year 2, respectively. However, the 
proportion of individuals employed each year began to decline in Year 3. In that year, 84 per- 



'^Box 2.3 in Chapter 2 presents instructions on reading an impact table. 



The New Hope Project 
Table 3.1 

Impacts on Employment, Earnings, and Earnings-Related Income 
Over Five Years 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size“ 


Ever employed (%) 


Year 1 


90.1 


81.9 


8.2 *** 


0.001 


10.0 


0.23 


Year 2 


90.0 


82.7 


7.3 *** 


0.003 


8.9 


0.21 


Year 3 


84.0 


82.0 


2.0 


0.464 


2.4 


0.05 


Year 4 


83.2 


80.6 


2.6 


0.345 


3.2 


0.07 


Year 5 


81.1 


80.0 


1.2 


0.681 


1.5 


0.03 


Years 1 to 5 (annual average) 


97.2 


94.8 


2.5 * 


0.080 


2.6 


0.13 


Average annual number of quarters 
employed per year 


Year 1 


3.0 


2.6 


0.4 *** 


<.0001 


15.8 


0.27 


Year 2 


3.0 


2.7 


0.3 *** 


0.010 


10.0 


0.18 


Year 3 


3.0 


2.9 


0.1 


0.187 


5.1 


0.09 


Year 4 


2.9 


2.8 


0.1 


0.274 


4.4 


0.08 


Years 


2.9 


2.8 


0.0 


0.841 


0.8 


0.01 


Years 1 to 5 (annual average) 


3.0 


2.8 


0.2 ** 


0.022 


7.0 


0.16 


Average annual earnings ($) 


Year 1 


7,007 


5,919 


1,088 *** 


0.003 


18.4 


0.17 


Year 2 


8,310 


7,886 


424 


0.368 


5.4 


0.06 


Year 3 


9,617 


8,897 


720 


0.194 


8.1 


0.09 


Year 4 


10,417 


9,966 


451 


0.471 


4.5 


0.05 


Years 


11,324 


10,824 


500 


0.466 


4.6 


0.05 


Years 1 to 5 (annual average) 


9,335 


8,698 


637 


0.159 


7.3 


0.09 


Average annual eamings-related 
income (earnings, EITC, 
and supplement) ($) 


Year 1 


8,558 


6,943 


1,615 *** 


<.0001 


23.3 


0.24 


Year 2 


10,169 


9,200 


969 * 


0.057 


10.5 


0.12 


Year 3 


11,799 


10,404 


1,395 ** 


0.018 


13.4 


0.16 


Year 4 


12,017 


11,504 


513 


0.434 


4.5 


0.05 


Years 


12,924 


12,309 


614 


0.390 


5.0 


0.06 


Years 1 to 5 (annual average) 


11,093 


10,072 


1,021 ** 


0.033 


10.1 


0.14 


Sample size 


366 


379 


— 









SOURCES: MDRC calculations using data from the New Hope Project MIS client- tracking database and 
Wisconsin unemployment insurance (UI) records. 



NOTES: A two-tailed t-test was used to assess the statistical significance of each difference in characteristics 
between the program and control groups. Statistical significance levels are indicated as *** = 1 percent, ** = 5 
percent, and *=10 percent. 

“The effect size is the difference between program and control group outcomes expressed as a proportion of the 
standard deviation of the outcome for both groups combined. This standard deviation is always obtained from the 
entire sample, even if the table shows impacts for subgroups. 
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cent of program group members were employed, compared with 82 percent of control group 
members — an insignificant difference of 2 percent. While Figure 3.2 and the top panel of Ta- 
ble 3.1 show that employment among both the program group and the control group declined, 
the table reveals a steeper decline in employment among program group members after Year 3. 

Consistent with Figure 3.2, the impacts on the average number of quarters employed were 
largest in Years 1 and 2; the program group worked an average of three quarters per year during 
the first two years, compared with slightly less than three quarters for the control group. These 
differences disappeared in Year 3, the last year of the program. Over the entire five-year period, 
however, the difference in the number of quarters employed per year was statistically significant, 
with New Hope participants working more quarters than their control group counterparts. 

Employment and earnings for the New Hope group include community service jobs 
(CSJs), or jobs available to people who wanted full-time employment but were unable to find it 
on their own. Analysis presented in the two-year report suggests that the CSJs accounted for a 
substantial part of New Hope’s employment effects. However, the effects of CSJs cannot be 
separated from the effects of other New Hope components, because there was no group in the 
study that had access to New Hope’s financial incentives and work supports but not to the CSJ 
component. In other words, since all New Hope program group members could use all parts of 
the program, there is no valid measure of what CSJ users would have done in the absence of this 
program component. However, nonexperimental comparisons can be made among program 
group members who did and did not participate in CSJs (see Box 3.2). 

The middle rows of Table 3. 1 show that New Hope’s impacts on average annual earnings 
were large in the first year but faded over the remaining period. For example, in the first year, av- 
erage aimual earnings were 1 8 percent higher for the program group — or $7,007, compared with 
$5,9 1 9. Although earnings increased over time for both groups, the program group did not match 
the jump in earnings experienced by the control group in Year 2 and later periods. As a result, the 
earnings differences for Years 2 through 5 were smaller and not statistically significant. 

The bottom rows of Table 3.1 show impacts on eamings-related income (that is, the 
sum of earnings, EITC benefits, and the New Hope supplement). New Hope’s impacts were 
positive and significant for the first three years of the program, increasing eamings-related in- 
come by 10.5 percent to 23.3 percent each year. The differences were still positive across Years 
4 and 5 but were not statistically significant at conventional levels. This suggests that the New 
Hope supplement may not provide long-term impacts on employment-related income if it is 
implemented with a stop-point, as in this demonstration.'^ However, the average impact on 



‘^The original design of New Hope was intended to extend supplemental payments indefinitely. 
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Box 3.2 



Community Service Jobs Led to Stable Employment 
for Some New Hope Participants but Not for Others 

Community service jobs (CSJs) offered some of the participants in the New Hope 
Ethnographic Study httle more than a chance to accumulate enough work hours to be able to use 
New Hope’s benefits, and these participants often cycled between several CSJs while enrolled 
in New Hope. For many other famihes, a CSJ position helped participants to find a better 
employment pathway and even to secure long-term, stable employment and wage growth. 

For example, Anne, an Afiican-American single mother of two elementary-school- 
aged children, worked at three different CSJs over the course of enrollment in New Hope. Her 
first placement was at Goodwill Industries. This CSJ involved a half-day of training and a half- 
day of factory work with the disabled. Aime enjoyed working with and talking with the disabled 
people who worked there, but she really disliked the training. It was supposed to last only three 
or four months, but because a teacher quit halfway through, the program had to hire another 
teacher, and the training ended up lasting six months. According to Anne, the new teacher was 
not very good; this person “just sat there and ate” but did not really teach much. The course was 
supposed to teach basic skills, and Anne felt that it was geared more toward people who wanted 
a General Educational Development (GED) certificate, not high school graduates like herself 
Anne’s second CSJ placement was at a daycare center. She did not like this placement because 
her CO workers seemed unhappy: “I think that people should like their work. The people there 
would just sit around and complain about how long it was until they would get off Then when 
the kid’s parents came around, they put on this big act.” She quit the placement. Her third CSJ 
was at a nonprofit organization downtown that worked with people who have been arrested. Her 
duties included reception and escorting people fi-om prison to the office. She liked these duties 
for the most part (she did not like going to the prison, because it smelled bad). While this place- 
ment did not become permanent, it did help her to realize what sort of work environment she 
preferred. She liked the professional atmosphere of what she called a “downtown job.” Some of 
the counselors at this job impressed her as role models. She said: “They were my age, or some 
were even younger, but they all had their bachelor’s degree and this job. They seemed very 
comfortable [financially].” For the few years after New Hope ended, Anne worked several jobs 
in professional offices as either a secretary or an administrative assistant. Her earnings did in- 
crease while she was enrolled in New Hope — fi'om nearly S3,000 in the first year to just over 
$6,100 in her final year of ehgibihty. However, by the fifth year, her earnings had drifted down 
to about $4,600. Thus, while the third CSJ placement gave Arme a better idea of the kind of 
work she preferred, she was unable to find stable employment or wage growth during and after 
her involvement with New Hope. 

L’Kesha, an African-American woman aged 28, was featured in the two-year New 
Hope report. Her CSJ placement in a shelter for homeless men turned into a full-time regular 
position that was stable for the duration of the New Hope program and the ethnographic visits 
that followed. Prior to enrolling in New Hope, L’Kesha was receiving AFDC, had no em- 
ployment experience, had not finished high school or received her GED, and had three small 
children to care for. The CSJ at the shelter afforded her the opportunity to learn valuable skills 
like typing and knowledge of office software on the computer. After becoming a regular em- 
ployee, she received benefits and was promoted a couple of times over the course of several 
years. L’Kesha’s earnings increased sharply during this period, from just over $3,800 in her 
first year of New Hope to more than $14,600 five years after her enrollment. 
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earnings-related income over the five-year period was large ($1,021, or 10 percent) and statisti- 
cally significant. 

Effects on Benefit Receipt 

Table 3.2 documents program effects on welfare receipt, food stamp receipt, and total 
income. While the percentage of program group members who received welfare (that is, AFDC 
or W-2) declined dramatically over time. New Hope had no effect on benefit usage. In other 
words, welfare receipt declined dramatically over the study period for both groups. The table 
also shows that New Hope had no significant effects on the dollar amount of welfare or food 
stamp receipt. For example, in the first year, average aimual welfare benefits received were 
$3,496 for the program group and $3,583 for the control group. By Year 5, these numbers had 
fallen to $476 and $466, respectively. The implementation of W-2 may have influenced the be- 
havior of both program group and control group members, by making it more difficult and less 
desirable to remain on the welfare rolls. 

Effects on Total Income 

The next panels of Table 3.2 present impacts on total income and poverty. Total income 
includes earnings, EITC benefits, the New Hope supplement, welfare, and food stamps. New 
Hope increased average annual total income during the entire follow-up period by about 7 per- 
cent, or $883 ($14,039 for the program group, compared with $13,156 for the control group). 
As with the impacts on earnings-related income, most of the impacts on total income occurred 
during Years 1 through 3, or while the program was still in effect. The impacts in Years 4 and 5 
remained positive but were not statistically significant. Box 3.3 discusses the relative contribu- 
tion of each factor of income to the increase over the entire period. 

Poverty rates are calculated by comparing annual income for each family with the pov- 
erty line appropriate for that family’s size. The last rows of Table 3.2 show a measure of “se- 
vere” poverty, defined as income below 50 percent of the poverty line. Note that since this 
measure of poverty is based on income that is calculated from administrative records and does 
not include other sources of household income (such as other benefits and the earnings of other 
adults), it is not directly comparable to the official poverty rate. New Hope substantially reduced 
poverty over the entire follow-up period, and the effects were equally strong in both the early 
and the later periods. Over the entire period, for example, 52.7 percent of the program group 
had incomes below the poverty line, compared with 66.3 percent of the control group. New 
Hope also significantly reduced severe poverty in Years 1 and 2, which translated to a reduction 
over the entire period. For example, over the entire period, 13 percent of the program group ex- 
perienced incomes below 50 percent of the poverty line, compared with almost 20 percent of 
the control group — a significant impact of nearly 7 percent. 





The New Hope Project 
Table 3.2 



Impacts on Benefit Receipt and Total Income Over Five Years® 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size^ 


Ever received AFDC/TANF (%) 


Year I 


81.9 


81.6 


0.3 


0.899 


0.3 


0.01 


Year 2 


58.4 


61.9 


-3.5 


0.270 


-5.6 


-0.07 


Year 3 


34.0 


36.3 


-2.3 


0.494 


-6.3 


-0.05 


Year 4 


18.8 


19.3 


-0.6 


0.842 


-2.9 


-0.01 


Year 5 


13.7 


15.0 


-1.3 


0.616 


-8.4 


-0.04 


Years 1 to 5 (annual average) 


83.8 


84.8 


-1.1 


0.603 


-1.2 


-0.03 


Average annual amount of 
AFDC/TANF received ($) 


Year 1 


3,496 


3,583 


-87 


0.576 


-2.4 


-0.03 


Year 2 


1,976 


2,215 


-240 


0.152 


-10.8 


-0.10 


Year 3 


977 


1,116 


-139 


0.316 


-12.5 


-0.07 


Year 4 


659 


670 


-11 


0.930 


-1.6 


-0.01 


Year 5 


476 


466 


10 


0.922 


2.2 


0.01 


Years 1 to 5 (annual average) 


1,517 


1,610 


-93 


0.328 


-5.8 


-0.06 


Average annual amount of 
food stamps received ($) 


Year 1 


2,229 


2,211 


18 


0.839 


0.8 


0.01 


Year 2 


1,711 


1,669 


42 


0.681 


2.5 


0.03 


Year 3 


1,221 


1,338 


-117 


0.284 


-8.7 


-0.07 


Year 4 


1,053 


1,148 


-95 


0.376 


-8.3 


-0.06 


Year 5 


929 


1,002 


-72 


0.468 


-7.2 


-0.05 


Years 1 to 5 (annual average) 


1,429 


1,474 


-45 


0.557 


-3.0 


-0.04 


Average annual total income^ ($) 


Year 1 


14,283 


12,737 


1,546 ♦♦♦ 


<.0001 


12.1 


0.25 


Year 2 


13,856 


13,084 


772 


0.115 


5.9 


0.11 


Year 3 


13,997 


12,858 


1,139 ♦♦ 


0.049 


8.9 


0.14 


Year 4 


13,729 


13,322 


407 


0.524 


3.1 


0.05 


Year 5 


14,329 


13,777 


552 


0.430 


4.0 


0.06 


Years 1 to 5 (annual average) 


14,039 


13,156 


883 ♦ 


0.058 


6.7 


0.13 


Total income below the poverty 
standard‘s (%) 


Year 1 


52.7 


70.2 


-17.5 ♦♦♦ 


<.0001 


-24.9 


-0.36 


Year 2 


53.0 


62.5 


-9.5 ♦♦ 


0.021 


-15.2 


-0.19 


Year 3 


50.9 


65.6 


-14.7 ♦♦♦ 


0.000 


-22.4 


-0.30 


Year 4 


53.8 


62.4 


-8.6 ♦♦ 


0.038 


-13.7 


-0.17 


Year 5 


52.2 


60.1 


-7.9 ♦ 


0.055 


-13.2 


-0.16 


Years 1 to 5 (annual average) 


52.7 


66.3 


-13.5 ♦♦♦ 


0.001 


-20.4 


-0.28 



(continued) 
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Table 3.2 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size'’ 


Total income below 50% of poverty 
Standard‘S (%) 

Year 1 


11.8 


18.2 


-6.4 ** 


0.030 


-35.3 


-0.18 


Year 2 


15.0 


23.3 


-8.3 ** 


0.012 


-35.7 


-0.21 


Years 


21.2 


22.3 


-1.1 


0.750 


-5.0 


-0.03 


Year 4 


20.9 


26.8 


-5.9 


0.104 


-22.0 


-0.14 


Year 5 


22.9 


28.3 


-5.4 


0.148 


-19.2 


-0.12 


Years 1 to 5 (annual average) 


13.0 


19.9 


-6.9 ** 


0.028 


-34.8 


-0.2 


Sample size 


366 


379 











SOURCES: MDRC calculations using data from the New Hope Project MIS client-tracking database, Wisconsin 
unemployment insurance (UI) records, and Wisconsin Department of Workforce Development AFDC and Food 
Stamp records. 



NOTES: A two-tailed t-test was used to assess the statistical significance of each difference in characteristics 
between the program and control groups. Statistical significance levels are indicated as *** = 1 percent, ** = 5 
percent, and * = 10 percent. 

^otal income is calculated as the sum of earnings, EITC benefits, New Hope supplements, welfare benefits, 
and food stamps. 

'’The effect size is the difference between program and control group outcomes expressed as a proportion of 
the standard deviation of the outcome for both groups combined. This standard deviation is always obtained from 
the entire research sample, even if the table shows impacts for subgroups. 

‘^Poverty measures are based on income that is calculated from administrative records and does not include 
other sources of household income. Measures are not directly comparable to the official poverty rate. 
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Box 3.3 

What Is the Primary Cause for the Increase in Total Income? 

Total income is calculated as the sum of earnings, Earned Income Tax Credits (EITCs), 
New Hope supplements, welfare benefits, and food stamps. The decomposition of the impacts 
on total income over the five-year period is shown in the accompanying figure. The figure and 
Table 3.1 confirm that the increase in total income in Year 1 appears to be primarily due to 
higher earnings among the program group. While there is no significant impact on total income 
in Year 2, the figure shows that the New Hope supplement became the largest contributor, con- 
sistent with the eamings-related income reported in the bottom panel of Table 3.1 . In Year 3, the 
increase in total income appears to be driven by the significant increase in EITC benefits among 
the program group. By Year 4 — when the New Hope program was no longer in effect — earn- 
ings, EITC benefits, welfare benefits, and food stamp benefits were similar between the pro- 
gram group and the control group. This impUes that New Hope increased total income through a 
variety of factors, not simply through its earnings supplement. 

The figure also reflects the increase in income for the control group over time, as 
shown in the tables. For example, in Year 1, New Hope program group members experienced 
a significant difference in total income of approximately $1,546. The control group earnings 
increased markedly in Year 2 (as indicated by the decline in the earnings impact in the figure 
and in Table 3.1), resulting in similar total income levels for both the program group and the 
control group. In fact, while earnings impacts were generally positive throughout the period, 
they were significantly higher only in the first year. This is illustrative of how the good econ- 
omy and the work requirements under W-2 may have combined to increase employment 
among the low-income population in Wisconsin. 



Decomposition of the Differences in Total Income Bet>veen the 
Program Group and the Control Group 
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* In Year 1 , food stamps is represented as the third bar from the bottom of the stack and has a value of 
$17.69. 





These large impacts on poverty may seem surprising, given the moderate increase in total 
income, particularly in the last three years. However, separate analyses (not shown) suggest that 
the decline in poverty stemmed in part from the fact that New Hope reduced the number of fami- 
lies with very low incomes. For example. New Hope significantly reduced the number of families 
with incomes between $7,000 and $1 1,000 — by 6 percent (not shown). Impacts such as these, on 
the distribution of income, do not always show up in a comparison of average incomes. 

Effects on Other Employment Outcomes 

Table 3.3 presents impacts on other aspects of emplo)mient, including employment sta- 
bility, wages, hours, and job benefits. The first two rows show that New Hope increased stable 
employment. Program group members were more likely to go to work during Year 1, and, 
among those who did, a large proportion stayed employed consistently. In other words, of the 
8.2 percentage point increase in employment in Year 1, 6.9 percentage points of it was stable 
emplo)mient. Stable employment provides people with a better chance of attaining economic 
security in future years than does sporadic employment. 

Despite the good news on employment stability and total income reported earlier, the 
data still show a fair amount of job loss. Only about 63 percent of program group and control 
group members reported that they were working full time as of the five-year survey interview. 
This is consistent with the overall decline in employment reported in the top panel of Table 3.1. 
These differences suggest that full employment may have been difficult to find in Milwaukee. 

The next several rows of Table 3.3 present data on wages and hours worked.'"* One re- 
sult of the increase in stable employment may have been an increase in wages for the program 
group. Program group members were more likely to have wages (in their current or most recent 
job as of the five-year survey) that exceeded $11 per hour — a difference of 6.6 percentage 
points. There were no significant differences between the program group and control groups in 
hours worked or in benefits offered at their most recent job. 



Employment Impacts for Subgroups 

As shown in many previous evaluations, including New Hope’s two-year evaluation, 
programs often have different effects for different subgroups within the sample. In fact, small 
impacts for the sample overall can often mask large impacts for particular subgroups, defined. 



'"'Each of these outcomes is calculated over the full sample, in order to preserve the experimental compari- 
son. Thus, the percentages for hours worked do not sum to 100 percent but instead to the percentage of the 
sample who worked at some point during the fifth year. 
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The New Hope Project 
Table 3.3 

Impacts on Job Characteristics Five Years After Random Assignment 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size“ 


Employment stability^ (%) 


Worked in Year 1 


90.1 


81.9 


8.2 ♦♦♦ 


0.001 


10.0 


0.23 


And worked more than 12 


consecutive quarters 


43.4 


36.5 


6.9 *♦ 


0.047 


18.8 


0.14 


Worked full time at 60-month 


survey (%) 


63.4 


63.3 


0.1 


0.977 


0.2 


0.00 


Hourly wage at most recent Job (%) 
Worked in Year 5 and hourly wage 


Less than $7 per hour 


16.6 


15.0 


1.6 


0.613 


10.7 


0.04 


Between $7 and $9 per hour 


24.5 


27.7 


-3.2 


0.408 


-11.5 


-0.07 


Between $9 and $1 1 per hour 


24.4 


29.9 


-5.5 


0.158 


-18.4 


-0.12 


More than $1 1 per hour 


26.6 


20.0 


6.6 * 


0.065 


33.2 


0.16 


Hours worked per week, at most 
recent job*^ (%) 


Less than 30 


12.2 


12.0 


0.2 


0.944 


1.7 


0.01 


30 to 45 


67.5 


71.7 


-4.2 


0.294 


-5.8 


-0.09 


More than 45 


12.8 


8.9 


3.9 


0.146 


43.5 


0.12 


Benefits offered at most recent Job (%) 


Sick days 


52.2 


59.0 


-6.8 


0.109 


-11.6 


-0.14 


Paid vacation 


63.9 


66.0 


-2.1 


0.607 


-3.2 


-0.04 


Health insurance 


64.4 


67.6 


-3.1 


0.444 


-4.7 


-0.07 


Pension 


50.6 


55.3 


-4.6 


0.279 


-8.4 


-0.09 


Sample size 


281 


276 











SOURCES: MDRC calculations using data from the New Hope Background Information Form (BIF) and 
five-year survey. 



NOTES: The information on job characteristics is only available for the 60-month survey. 

A two-tailed t-test was used to assess the statistical significance of each difference in characteristics 
between the program and control groups. Statistical significance levels are indicated as *** = 1 percent, ** = 
5 percent, and *=10 percent. 

“The effect size is the difference between program and control group outcomes expressed as a proportion 
of the standard deviation of the outcome for both groups combined. This standard deviation is always 
obtained from the entire research sample, even if the table shows impacts for subgroups. 

^Employment stability is calculated for the entire CFS sample (total = 745). 

‘^Each of these outcomes is calculated over the entire sample, in order to preserve the experimental 
comparison. Thus, the percentages for hours worked do not sum to 100 percent but, instead, to the percentage 
of the sample who worked at some point during the fifth year. 
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for example, by education level or employment status. This section presents New Hope’s effects 
on summary measures of employment and earnings for several key subgroups. 

Effects by Employment Status at Random Assignment 

The New Hope treatment was viewed very differently by sample members who were 
already employed full time at random assignment than by members who had to increase their 
work effort to meet the 30 hours of work per week that were required in order to qualify for 
New Hope benefits. In the two-year evaluation, the impacts on employment and income were 
different for those who were employed full time at baseline and those who were not. 

The first two panels of Table 3.4 report employment and earnings impacts by employ- 
ment status at random assignment. Consistent with findings from the two-year report, the sec- 
ond panel shows that New Hope increased employment in the first year among people who 
were not employed full time at random assignment.'^ In the first year, for example, the program 
group worked an average of 2.7 quarters each year, compared with 2.2 quarters for the control 
group. Among those not employed full time at baseline, earnings for the program group were 
higher in Year 1, by 28 percent.'® In contrast, the program had few significant effects for the 
group employed full time at baseline. An interesting pattern is that employment (measured by 
quarters employed per year) actually fell somewhat for the groups employed full time at base- 
line, for both the program and the control groups. The rightmost column of the table presents 
the p-value for a test of whether the impacts for each subgroup were significantly different, that 
is, whether the impact on employment in Year 1 for those employed full time at baseline was 
significantly different from the corresponding impact for those not employed full time at base- 
line. As the numbers indicate, the only statistically significant difference in impacts was for em- 
ployment in Year 1 . 

Although not shown in the table, the impacts on employment stability and hourly wages 
were also concentrated among the sample that was not fully employed at random assignment. 
New Hope increased stable employment for this group and also increased the number of people 
earning wages above $ 1 1 per hour. As mentioned, the higher wages earned by the program 

'®The rightmost column of Table 3.4 shows the results of a test that assesses whether the variation in im- 
pacts across the subgroups shown was statistically significant. In other words, was the effect on employment 
for those employed full time at random assignment significantly larger than the impact for those who were not 
employed full time? The p-value of 0.014 for this outcome indicates that there was a 1.4 percent chance that 
the 2.7 percent gain for those employed full time and the 25.3 percent gain for those not employed full time 
were really the same impact. That is, the program truly had a differential impact. The inclusion of daggers indi- 
cates a significant difference, while a lack of daggers indicates that both subgroups experienced essentially the 
same impact on this outcome. 

'^However, this impact does not significantly differ by employment status at baseline, as indicated by the 
p-value in the rightmost column of the table. 
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The New Hope Project 
Table 3.4 

Impacts on Employment and Earnings Over Five Years for Selected Subgroups 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size® 


P- Value for 
Difference 
Across Panels'’ 


Emoloved full time at baseline 
















Average annual number of 
















quarters employed per year 
















Year 1 


S.S 


S.4 


0.1 


O.SOS 


2.7 


0.06 


0.014 tt 


Year 2 


S.S 


S.S 


0.1 


0.461 


S.S 


0.08 


0.276 


Years 


S.S 


S.S 


0.1 


0.78S 


l.S 


O.OS 


0.S76 


Year 4 


S.2 


S.l 


0.1 


0.664 


2.7 


O.OS 


0.778 


Years 


s.o 


S.l 


-0.1 


0.S8S 


-S.7 


-0.07 


0.420 


Years 1 to 5 (annual average) 


S.S 


S.S 


0.0 


0.7S0 


1.4 


0.04 


0.220 


Average annual earnings ($) 
















Year 1 


10,8S0 


10,0SS 


81S 


0.267 


8.1 


O.IS 


0.668 


Year 2 


11,620 


11,SSS 


26S 


0.767 


2.S 


0.04 


0.846 


Years 


12,S00 


11,9SS 


S47 


0.74S 


2.9 


0.04 


0.66S 


Year 4 


1S,S68 


1S,S07 


61 


0.961 


O.S 


0.01 


0.670 


Years 


14,S2S 


14,247 


78 


0.9SS 


0.6 


0.01 


0.66S 


Years 1 to S (annual average) 


12,49S 


12,179 


SIS 


0.726 


2.6 


O.OS 


0.642 


Sample size 


112 


110 












Not emoloved full time at baseline 














Average annual number of 
















quarters employed per year 
















Year 1 


2.7 


2.2 


0.6 *** 


<.0001 


2S.S 


0.S7 




Year 2 


2.8 


2.S 


O.S ** 


O.OIS 


1S.7 


0.2S 




Years 


2.9 


2.7 


0.2 


0.1 9S 


6.6 


0.12 




Year 4 


2.8 


2.7 


0.2 


0.276 


S.8 


0.10 




Years 


2.8 


2.7 


0.1 


O.SSl 


S.S 


O.OS 




Years 1 to S (annual average) 


2.8 


2.S 


O.S ** 


O.OIS 


lO.S 


0.21 




Average annual earnings ($) 
















Year 1 


S,S69 


4,194 


1,17S *** 


O.OOS 


28.0 


0.19 




Year 2 


6,904 


6,4SS 


470 


0.402 


7.S 


0.06 




Years 


8,497 


7,607 


890 


—0.-174 


11.7 


0.11 




Year 4 


9,208 


8,SS6 


672 


0.SS6 


7.9 


0.08 




Years 


10,112 


9,S46 


76S 


O.SSS 


8.2 


0.08 




Years 1 to S (annual average) 


8,018 


7,22S 


794 


O.ISO 


11.0 


0.12 




Sample size 


2S4 


268 
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Table 3.4 (continued) 





Program 


Control 


P-Value for 


% 


Effect 


P-Value for 
Difference 


Outcome 


Group 


Group 


Difference Difference 


Impact 


Size" 


Across Panels^ 



No potential barriers^ 



Average annual number of 
quarters employed per year 



Year 1 


3.0 


2.9 


0.2 


0.389 


S.3 


0.10 


0.190 




Year 2 


3.1 


3.1 


0.0 


0.832 


1.3 


0.03 


0.276 




Year 3 


3.1 


3.0 


0.1 


0.S49 


4.4 


0.08 


0.323 




Year 4 


2.9 


2.9 


0.0 


1.000 


0.0 


0.00 


0.4S7 




Years 


2.9 


2.9 


0.0 


0.826 


1.7 


0.03 


0.407 




Years 1 to 5 (annual average) 


3.0 


2.9 


0.1 


0.6S1 


2.S 


0.06 


0.4 IS 




Average annual earnings ($) 


















Year 1 


7,766 


7,660 


106 


0.880 


1.4 ■ 


0.02 


0.072 


t 


Year 2 


8,892 


9,72S 


-833 


0.3S1 


-8.6 


-0.11 


0.040 


tt 


Year 3 


10,S10 


10,128 


382 


0.729 


3.8 


O.OS 


0.008 


ttt 


Year 4 


11,310 


11,040 


269 


0.831 


2.4 


0.03 


0.098 


t 


Years 


11,821 


11,748 


73 


0.9S8 


0.6 


0.01 


0.117 




Years 1 to S (annual average) 


10,060 


10,060 


-1 


1.000 


0.0 


0.00 


0.019 


tt 


Sample size 


108 


104 















One potential barrier^ 



Average annual number of 
quarters employed per year 



Year 1 


3.0 


2.6 


0.4 


** 


0.014 


1S.2 


0.26 


Year 2 


3.1 


2.7 


0.4 


*** 


0.008 


16.4 


0.29 


Year 3 


3.2 


2.9 


0.3 


* 


0.08S 


10.1 


0.19 


Year 4 


3.1 


2.8 


0.3 


* 


0.08S 


10.9 


0.19 


Years 


2.9 


2.7 


0.2 




0.284 


7.2 


0.12 


Years 1 to S (annual average) 


3.0 


2.7 


0.3 


** 


O.OIS 


11.9 


0.26 


Average annual earnings ($) 
















Year 1 


7,801 


S,747 


2,0S4 


*** 


0.001 


3S.7 


0.33 


Year 2 


9,S4S 


7,77S 


1,770 


** 


0.019 


22.8 


0.24 


Year 3 


11,014 


8,670 


2,34S 


*** 


0.007 


27.0 


0.30 


Year 4 


11,S13 


9,736 


1,777 


* 


0.066 


18.3 


0.20 


Years 


12,706 


10,67S 


2,032 


* 


0.0S7 


19.0 


0.21 


Years 1 to S (annual average) 


10,S16 


8,S20 


1,996 


*** 


O.OOS 


23.4 


0.29 


Sample size 


161 


1S7 
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Table 3.4 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size" 


P-Value for 
Difference 
Across Panels^ 


Two potential barriers or more 
















Average annual number of 
















quarters employed per year 
















Year I 


2.9 


2.S 


0.6 


0.001 


27.9 


0.42 




Year 2 


2.7 


2.5 


0.2 


0.S24 


8.6 


0.14 




Years 


2.7 


2.8 


-0.1 


0.554 


-4.8 


-0.09 




Year 4 


2.8 


2.8 


0.0 


0.946 


0.6 


0.01 




Year 5 


2.7 


2.9 


-0.2 


0.S97 


-6.4 


-0.12 




Years 1 to 5 (annual average) 


2.8 


2.7 


0.1 


0.510 


4.1 


0.09 




Average annual earnings ($) 
















Year 1 


5,016 


4,521 


496 


0.425 


11.0 


0.08 




Year 2 


5,802 


6,S42 


-540 


0.52S 


-8.5 


-0.07 




Years 


6,452 


8,227 


-1,776 * 


0.081 


-21.6 


-0.22 




Year 4 


7,860 


9,S02 


-1,441 


0.209 


-15.5 


-0.16 




Year 5 


8,825 


10,098 


-1,27S 


0.298 


-12.6 


-O.IS 




Years 1 to 5 (annual average) 


6,791 


7,698 


-907 


0.265 


-11.8 


-O.IS 




Sample size 


97 


118 












AFDCAV-2 or food stamp recipients 














Average annual number of 
















quarters employed per year 
















Year 1 


2.8 


2.S 


0.5 ♦♦♦ 


<.0001 


24.4 


0.S7 


0.029 tt 


Year 2 


2.9 


2.5 


0.4 ♦♦♦ 


0.004 


15.2 


0.26 


0.28S 


Years 


2.9 


2.8 


0.2 


0.156 


7.0 


0.12 


0.649 


Year 4 


2.9 


2.7 


0.2 


0.170 


7.1 


0.12 


0.562 


Year 5 


2.8 


2.8 


0.0 


0.7S5 


1.7 


O.OS 


0.981 


Years 1 to 5 (annual average) 


2.9 


2.6 


O.S ♦♦♦ 


0.009 


10.5 


0.22 


O.S 19 


Average annual earnings ($) 
















Year 1 


5,767 


4,449 


1,S18 ♦♦♦ 


0.002 


29.6 


0.21 


0.465 


Year 2 


7,064 


6,420 


644 


0.218 


10.0 


0.09 


0.850 


Year S 


8,625 


7,506 


1,119 ♦ 


0.068 


14.9 


0.14 


0.5S5 


Year 4 


9,S15 


8,655 


661 


0.S46 


7.6 


0.07 


0.880 


Year 5 


10,480 


9,75S 


727 


0.S55 


7.5 


0.07 


0.894 


Years 1 to 5 (annual average) 


8,250 


7,S57 


894 ♦ 


0.07S 


12.1 


O.IS 


0.701 


Sample size 


255 


262 
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Table 3.4 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size® 


P-Value for 
Difference 
Across Panels^ 


African-American. non-Hisoanic 














Average annual number of 
quarters employed per year 


Year 1 


3.1 


2.6 


O.S *** 


0.001 


17.S 


0.31 


0.98S 


Year 2 


3.0 


2.8 


0.3 ** 


0.037 


10.4 


0.19 


0.428 


Year 3 


2.9 


3.0 


-0.1 


0.S04 


-3.3 


-0.06 


o.oos ttt 


Year 4 


2.9 


2.9 


0.0 


0.960 


-0.3 


0.00 


0.123 


Year 5 


2.8 


2.9 


-0.1 


0.S28 


-3.4 


-0.06 


O.lSl 


Years 1 to 5 (annual average) 


2.9 


2.8 


0.1 


0.341 


3.8 


0.09 


0.092 t 


Average annual earnings ($) 


Year 1 


7,248 


S,813 


1,43S 


0.003 


24.7 


0.23 


0.133 


Year 2 


8,222 


7,63 S 


S87 


0.334 


7.7 


0.08 


0.684 


Year 3 


9,321 


8,930 


391 


0.S87 


4.4 


O.OS 


0.169 


Year 4 


10,036 


9,99S 


42 


0.9S9 


0.4 


0.00 


0.120 


Year 5 


11,229 


10,880 


349 


0.70S 


3.2 


0.04 


0.282 


Years 1 to 5 (annual average) 


9,211 


8,6S1 


S61 


0.341 


6.S 


0.08 


0.1S8 


Sample size 


210 


200 












Hispanic 

Average annual number of 
quarters employed per year 


Year 1 


2.9 


2.S 


0.4 ♦♦ 


0.03S 


17.1 


0.28 




Year 2 


3.1 


2.6 


O.S ♦♦ 


0.011 


21.4 


0.36 




Year 3 


3.2 


2.4 


0.8 ♦♦♦ 


0.001 


31.9 


O.SO 




Year 4 


3.0 


2.4 


O.S ♦♦ 


0.021 


22.0 


0.33 




Years 


2.9 


2.S 


0.4 ♦ 


0.084 


16.2 


0.2S 




Years 1 to 5 (annual average) 


3.0 


2.S 


O.S ♦♦♦ 


0.002 


21.7 


0.43 




Average annual earnings ($) 


Year 1 


7,643 


S,816 


1,827 ♦♦ 


O.OIS 


31.4 


0.29 




Year 2 


9,298 


8,112 


1,187 


0.219 


14.6 


0.16 




Year 3 


10,838 


8,236 


2,602 ♦♦ 


0.02S 


31.6 


0.32 




Year 4 


11,73S 


8,930 


2, 80S ♦♦ 


0.031 


31.4 


0.31 




Years 


12,621 


10,184 


2,437 ♦ 


0.079 


23.9 


0.2S 




Years 1 to S (annual average) 


10,427 


8,2S6 


2,171 ♦♦ 


0.020 


26.3 


0.31 




Sample size 


102 


116 
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Table 3.4 (continued) 





Program 


Control 


P-Value for 


% 


Effect 


P-Value for 
Difference 


Outcome 


Group 


Group 


Difference Difference 


Impact 


Size“ 


Across Panels^ 



White. non-Hispanic 



Average annual number of 
quarters employed per year 



Year 1 


2.8 


2.4 


0.4 


0.149 


18.0 


0.29 


Year 2 


S.O 


2.9 


0.1 


0.77S 


S.2 


0.06 


Years 


S.2 


S.O 


0.2 


0.S99 


S.O 


0.10 


Year 4 


S.l 


S.l 


0.0 


0.914 


-1.1 


-0.02 


Years 


2.9 


S.l 


-0.2 


0.SS2 


-6.6 


-O.IS 


Years 1 to 5 (annual average) 


S.O 


2.9 


0.1 


0.710 


S.O 


0.07 


Average annual earnings ($) 














Year 1 


S,971 


6,707 


-131 


O.SOS 


-11.0 


-0.12 


Year 2 


7,848 


8,186 


-SS8 


0.822 


-4.1 


-O.OS 


Years 


9,07S 


9,7S2 


-660 


0.691 


-6.8 


-0.08 


Year 4 


10,SSS 


11,6S6 


-1,10S 


O.SSl 


-9.S 


-0.12 


Years 


10,S04 


11,S00 


-996 


0.600 


-8.7 


-0.10 


Years 1 to S (annual average) 


8,786 


9,SS2 


-767 


0.S77 


-8.0 


-0.11 


Sample size 


S9 


S4 











SOURCES: MDRC calculations using data from the New Hope Background Information Form (BIF) and 
Wisconsin unemployment insurance (UI) records. 

NOTES: A two-tailed t-test was used to assess the statistical significance of each difference in character istics 
between the program and control groups. Statistical significance levels are indicated as *** = 1 percent, ** = 5 
percent, and *=10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

“The effect size is the difference between program and control group outcomes expressed as a proportion of 
the standard deviation of the outcome for both groups combined. This standard deviation is always obtained from 
the entire research sample, even if the table shows impacts for subgroups. 

^A statistical test was conducted to measure whether impacts presented for different groups in this table were 
significantly different from one another. This p-value represents the probability that apparent variation in impacts 
across different panels of the table is simply the result of random chance. If this probability is less than 10 percent, 
the variation in impacts is considered statistically significant. Statistical significance levels are indicated as ftt “ 1 
percent, ft = 5 percent, and f = 10 percent. 

‘^Potential barriers to employment are not having worked in the past six years; having been arrested since age 
1 6; having either two or more children under age 6 or four children under age 1 2; having been fired from one’s 
period of longest employment; and not having a GED or high school diploma. 
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group may reflect the fact that more of them worked steadily during the period (as indicated by 
the impacts on stable employment). 

Effects by Number of Barriers to Employment 

When local ethnographers interviewed New Hope participants and project staff, they 
uncovered significant variation in participants’ potential barriers to working full time and taking 
advantage of the New Hope offer.’’ On the basis of their observations, five potential barriers 
were identified: low level of education, responsibility for young children, an arrest record, lack 
of recent job experience, and having been fired fi-om one’s last job. Sample members were di- 
vided into three groups. The first group had none of the identified potential barriers to employ- 
ment. They were likely to be most ready to engage in full-time employment, so they may have 
needed relatively little assistance. Although they may have been helped by the New Hope Pro- 
ject, the resources available to the control group may have been sufficient to increase their em- 
ployment as well, thereby limiting the program’s potential effects on employment outcomes for 
the group without any potential barriers. 

The second group had one identified potential employment barrier that might be ad- 
dressed by the New Hope program. For example, this group includes families with several 
young children, for whom child care expense could have been a barrier to be addressed by the 
child care subsidy. Sample members who had been fired, who had no recent work experience, 
who had an arrest record, or who lacked a high school credential might benefit fi-om a CSJ to 
establish a work history. For this group, the New Hope offer might have been the “missing 
piece” needed to change their fortunes in the labor market. A third group, with multiple poten- 
tial bamers, might also have benefited fiom New Hope, but it would include sample members 
who had more serious impediments to employment and who needed an intervention more inten- 
sive than the one offered by New Hope. 

Table 3.4 compares these three groups to explore the question of whether the pattern 
suggested by a small number of ethnographic interviews is representative of the experiences for 
the sample as a whole. The impacts shown in the three panels supports the expectation that 
those sample members who had one barrier to employment would be affected the most by the 
program. For this group, the program increased annual earnings over the five-year period by 
23 .4 percent ($ 10,516 for the program group, compared with $8,520 for the control group). This 
gain was evenly distributed throughout the period. Although not shown in the table, for this 
group the program also increased stable employment (or the fiaction of the sample who started 
work in Year 1 and worked 12 consecutive quarters), increased incomes, and reduced poverty. 



’’Weisner et al., 1999. 
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The effects for those with many employment barriers were temporary and less dra- 
matic. While the program increased employment during Year 1, the impacts did not persist be- 
yond that point. Further, at the five-year follow-up point, program group members were less 
likely to have jobs offering benefits (not shown in table). Overall, New Hope had a temporary 
and very limited effect on those who had multiple barriers to employment. Box 3.4 illustrates 
several ways in which New Hope failed to help program group members who experienced mul- 
tiple barriers to employment. 

Effects by Public Assistance Status 

In the year preceding their application to New Hope, nearly three in four sample mem- 
bers reported receiving AFDC or food stamps. However, by no means did New Hope serve 
only welfare and food stamp recipients, as many other programs do. Chapter 2 shows that about 
one in four applicants to the program had never received welfare. Thus, New Hope was not de- 
signed or operated as a “welfare-to-work” program, intended to actively reduce welfare rolls. 
One might expect New Hope services and benefits to have affected public assistance receipt 
indirectly. Inasmuch as the program made work more attractive and employment a more feasi- 
ble alternative to receiving welfare, it may have enabled many families to reduce their reliance 
on public benefits. 

Such an effect would go hand in hand with the changing welfare environment in Wis- 
consin and in the United States as a whole. As New Hope was implemented, the State of Wis- 
consin embarked on a very ambitious overhaul of its welfare system, moving first to a program 
called Pay for Performance and then implementing Wisconsin Works (W-2) in 1997 (in effect, 
during the follow-up period covered by this report). Both of these programs required public as- 
sistance recipients to work or participate in employment activities in return for receiving a wel- 
fare grant. Helped by the strong economy, the results were remarkable: Welfare rolls in Wis- 
consin have fallen by 75 percent since 1991. In Milwaukee, caseload reductions were more lim- 
ited but still fairly large, at 38 percent. 

Given these large reductions in welfare receipt, the impact of New Hope might go two 
ways. On the one hand, the welfare changes provided program group members with an added in- 
centive to seek employment and leave public assistance. Therefore, one might have expected New 
Hope’s effects on welfare receipt to be strengthened. On the other hand, control group members 
also experienced increasing pressure from the welfare system, which, under W-2, was combined 
with increasingly generous and intensive services. These pressures on control group members 
could have offset New Hope’s effects on the behavior of those receiving public assistance. 

The sixth panel of Table 3.4 presents impacts for people who were receiving AFDC or 
food stamps at random assignment. Program group members worked more than control group 
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Box 3.4 



, New Hope Most Helped the Women 
Who Did Not Have Many Barriers to Employment 

Several of the women in the New Hope Ethnographic Study sample who had no 
employment barriers at baseline had stable employment and/or benefits at their place of 
employment and, as a result, were not as motivated to use New Hope benefits. For example, 
Julie, an African-American single parent with two teenage children, worked for a Ford dealer 
in Milwaukee as a service coordinator handling customer complaints when she applied for 
New Hope. During her enrollment period, Julie occasionally used New Hope’s health 
insurance, but she did so for only 10 months out of her 36-month eligibility period. And 
because her earnings generally put her over the poverty threshold, she only received an 
earnings suplement a few times during her enrollment, for about $20 each time. Finally, since 
both of her children were teenagers, she did not need the program’s child care support. 

New Hope was particularly helpful to families who had only one barrier to finding 
and keeping work, and it was even more helpful in cases where its benefits addressed with the 
kinds of barriers the women had. Edith, a married Latina mother of three young children, 
signed up for New Hope after her husband, Manuel, went to prison on a dmg charge. With 
Manuel away, Edith needed to work full time to make ends meet; and to work so many hours, 
she needed reliable child care — which is why she applied for New Hope. She had a stable 
job with health insurance benefits at a local social services agency. Edith used the child care 
support fi’om New Hope for the full 36 months of her eligibility. She appreciated the value of 
having been assigned to the program group and contrasted her good fortune to that of a close 
friend who had been assigned to the control group. Her friend also needed child care for her 
young son in order to accept a job that had been offered to her. But her fi-iend did not take the 
job because she did not have full-time care for her son. When Edith lost the New Hope child 
care support at the end of her period of eligibility, she came to appreciate its value all the 
more. Had the support not ended, her husband — who had returned home fi’om his incarcera- 
tion — could look for a part-time job, in addition to his current full-time job. The extra in- 
come that Manuel would bring in would have improved the family’s financial well-being. 
Edith tried to use the child care support fi’om W-2, but the family’s household income was 
over the income threshold, and they did not qualify. 

Many of the families with two or more employment barriers found it difficult to 
benefit fi’om the New Hope program because their needs were often too complex. For exam- 
ple, Beth, a Latina mother of four children, lived with her partner, Victor, who was the father 
of three of the children. Although Beth tried to become active in New Hope when she was 
first selected, she and Victor felt that the program could not meet their needs. Beth’s two main 
employment barriers were her young children, who needed child care, and her lack of ade- 
quate education. New Hope would provide child care, of course, but only if Beth worked 30 
hours or more a week, and she did not want to take just any job simply to get this benefit. She 
wanted to place her children in child care so that she could go to school full time and get a 
better education. Beth said that she “didn’t want to work at Taco Bell for five years and then 
look back to see that she hadn’t accomplished anything.” She wanted an education and a 
“good job.” Although New Hope offered to try to enroll Victor so that he could receive bene- 
fits, Victor could not receive the child care support, because he and Beth were not married. 
Eventually the two gave up on the program altogether. 

Ethnographic data also point to the ambiguity of terms like “barrier to employment.” 
Mothers, of course, do not classify their children as “barriers” to work, and the women in this 
study were forced to make the best choices they could for their family routine and their chil- 
dren. They were conflicted over their choices of working or being at home with their young 
children; ^ey often could not do everything that was expected, given the low-wage work that 
they so often faced — without benefits, without child care, and with very little flexibility. 
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members in the first two years. As a result, they also had higher average earnings during Year 1 : 
$5,767 for the program group, compared with $4,449 for the control group. The difference in 
employment rates diminished during Years 3 to 5, in part because of increased employment 
among the control group. Nonetheless, over the entire follow-up period, both earnings and em- 
ployment were significantly higher for the program group. Although not shown in the table. 
New Hope also increased stable employment and average wages for this subgroup. Impacts are 
not shown for those not receiving public assistance at baseline. On the whole, the impacts for 
this group were not significantly different from the impacts for the public assistance subgroup, 
as indicated by the p-values in the rightmost column, with the exception of the employment im- 
pact in Year 1. 

Effects by Race and Ethnicity 

The final three panels of Table 3.4 present effects by race and ethnicity. Significant 
variation is seen in employment impacts across three ethnic groups, with African-Americans 
and Hispanics working significantly more due to New Hope.'* In this case, the employment and 
earnings impacts were much more positive for African-American and Hispanic sample mem- 
bers than for their white counterparts. In fact, earnings impacts for the white sample were nega- 
tive, although statistically insignificant. 

Interestingly, Hispanics seemed to have benefited the most from the New Hope program. 
For example, the impacts on average earnings for Years 3 to 5 are much larger than the impacts in 
the first two years, even though the corresponding impacts on quarters of employment were simi- 
lar in both periods. This indicates that earnings were increasing over time at an increasing rate for 
the program group. The impacts on stable employment and on high wages were also somewhat 
larger for this group (not shown in table). Not surprisingly, the program’s effects on poverty were 
also larger for the African-American and Hispanic samples (not shown in table). 



Conclusion 

The New Hope program operated for three years, during which time participants could 
receive a range of work supports. It is perhaps not surprising, then, that the program’s effects oc- 
curred largely during that three-year period and faded thereafter. New Hope’s effects on employ- 
ment and earnings, for example, were large and statistically significant in the first few years. 

Although New Hope’s effects on earnings and income persisted throughout the five- 
year period for certain subgroups, there is some evidence of lasting effects for the sample as a 
whole. The program’s impacts on earnings, for example, were fairly large in the later years of 

'*No impacts are presented for other ethnic groups, because of small sample sizes. 
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follow-up, although not statistically significant. These effects contributed to the statistically sig- 
nificant reductions in poverty that occurred throughout the five-year period. In addition, the 
program group was earning higher wages at the five-year point — an effect that was probably 
due in part to the fact that they worked more consistently during the follow-up period. Thus, 
New Hope did have some effects beyond the time period during which it operated. 
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Chapter 4 

New Hope’s Effects on Parents’ Well-Being 



This chapter presents the impacts of New Hope on five domains of parents’ well-being: 
material well-being, residential well-being, physical well-being, instrumental behaviors and 
coping strategies, and psychosocial well-being. 



Key Findings 

• Overall, there were few differences between program and control group 
families. What few impacts there were often depended on parents’ status at 
baseline — when they applied to the New Hope Project — such as whether 
or not they were employed full time, the number of certain specific barriers 
to employment they had, and their ethnicity. Findings for subgroups defined 
at baseline were often stronger than those for the sample as a whole, despite 
reduced sample sizes for the subgroups. However, significant impacts some- 
times varied in direction for different subgroups. This picture is consistent 
with the view of New Hope as an intervention through which participants 
charted their individual courses toward heterogeneous outcomes. 

• Program and control group members reported similar levels of material well- 
being and residential well-being five years after applying to the project. Pro- 
gram group members who had been employed full time at baseline were 
more likely than control group members to be providing financial assistance 
to individuals outside their household (other than child support), and they had 
moved more often in the past three years. Among those not employed full 
time at baseline, program group members were less likely than controls to be 
providing financial assistance to others, and they were more satisfied with 
their neighborhood resources. 

• Program group members — especially those not employed full time at base- 
line — rated their physical health as better than did control group members. 

• In the domain of instrumental behaviors and coping strategies, members of 
the program group were more aware of community resources that could pro- 
vide, for example, advice about raising children or assistance with energy 
costs. Program group members demonstrated greater familiarity with the 
Earned Income Tax Credit (EITC) than controls, as well as a tendency to- 
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ward greater use of the EITC — although the latter effect was not statistically 
significant. 

• In the domain of psychosocial well-being, program group members reported 
fewer symptoms of depression than did control group members, but there 
were no differences in several other indicators of psychological well-being. 



New Hope’s Conceptual Framework 

As described in Chapter 1 (Figure 1.1), New Hope’s overall conceptual fi’amework pre- 
dicted impacts on parents’ employment and income, which, in turn, were expected to affect par- 
ents’ well-being. This chapter examines parents’ well-being in five domains: material, residen- 
tial, health, coping, and psychosocial. 

During the period of eligibility for program participation, benefits to participants may 
have begun with the “empowerment” of being able to choose the specific New Hope supports 
they wished to tap, facilitated by New Hope staff Availability of the four core New Hope bene- 
fits could be expected to improve material and residential well-being through their impacts on 
employment and income. Access to health insurance could be expected to improve health. The 
combination of available benefits could have contributed to psychosocial well-being by easing 
worries across a variety of domains and making the daily routine of family life more “sustain- 
able” and less vulnerable in the face of difficulties. 

Supports fi-om New Hope staff could go beyond assistance in finding employment, 
child care, and other resources; New Hope might also promote participants’ skills to “hold 
things together” as low-income working parents. Weisner and others determined fi’om their eth- 
nographic work with a subset of families in the New Hope sample that the families’ “carrying 
capacity” was often low, in terms of the resources, supports, and buffers available to help them 
cope with life’s contingencies.' The ethnographers viewed this largely as a consequence of the 
uncertainties and lack of benefits associated with low-wage employment. The benefits and sup- 
ports provided by New Hope, however, could help to keep these inevitable periodic difficulties 
fi’om cascading into more serious problems. The availability of a cafeteria of benefits fiom 
which to choose, coupled with the individual attention of New Hope staff members, could help 
to make family life more sustainable. 

The five-year survey took place two years after program participants’ eligibility for 
New Hope ended. What long-term effects on program participants could be expected, especially 



'Weisner etal., 1999,2000. 
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given the fact that the original New Hope model was that of an ongoing program, not one with 
time limits? And how might such impacts occur? 

First, especially for participants not already working full time at baseline. New Hope 
provided a supportive environment for developing the necessary routines of work, including 
making provisions for child care. Second, with the assistance of project staff, participants could 
learn to navigate bureaucracies and benefits in the public sphere, a prime example being the 
EITC.^ Third, those who worked gained potentially marketable job experience as well as evi- 
dence — for themselves and others — of their ability to hold a job and earn a living. Perhaps 
they also developed more “staying power” or employment stability, as suggested by their 
greater job stability (see Chapter 3). Working at a paid job could also establish a positive feed- 
back cycle; The social desirability of paid work, feelings of accomplishment, and the stimula- 
tion of learning new tasks and associating with a wider circle of people, could improve feelings 
of well-being, in turn buttressing the individual against difficulties encountered with working or 
in combining work and family responsibilities. Paid work, of course, also provides income and, 
in many cases, fringe benefits like health insurance — both contributing to material, health, and 
psychological well-being. In short, participants’ competencies, once acquired, and their expecta- 
tions, once established, could have continued beyond the life of the program. 



Review of the Two-Year Findings 

Two years into the New Hope program — and one year before participants’ eligibility 
ended — program group members enjoyed several advantages over their control group counter- 
parts. New Hope increased family income for those not employed full time at random assign- 
ment, and it reduced material hardship, mostly by increasing access to medical and dental care 
and by reducing periods without health insurance. The program group also reported signifi- 
cantly fewer financial worries than the control group. (Box 4.1 gives an example of the pathway 
from material well-being to psychosocial well-being and improved parenting.) 

At the two-year follow-up. New Hope had no impacts on residential well-being — par- 
ticipants’ home ownership, household formation, residential stability, money spent on housing, 
reported housing deficiencies, or satisfaction with housing. 

Program group members reported increased feelings of agency and hope (confidence 
that they could take action and achieve their goals), compared with those not in New Hope. 
However, there was no impact on a related measure: mastery. Parents in New Hope experienced 



^The use of EITCs was an integral component of the New Hope offer, with wage supplements being cal- 
culated on the assumption that participants would avail themselves of the state and federal tax credits for which 
they were eligible. 
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Box 4.1 



Material Well-Being Can Lead to Psychosocial Well-Being 
and Improved Parenting 

Many parents in the New Hope Ethnographic Study sample were explicitly aware of 
the cotmection between their sense of material security and their psychosocial well-being. For 
some, New Hope’s supports not only provided financial benefits and extra incentives to work 
but also relieved stress and made them feel happier. The financial relief allowed parents to 
buy things that their children wanted or to participate in enjoyable recreational activities. This 
is not simply mundane materialism. A number of parents said that such activities gave them a 
sense of personal satisfaction and belonging to their community. Part of feeUng like a good 
parent involved being able to take their children out to eat fast food, to enjoy a day at the mall, 
or to take a family vacation. 

Shortly after exiting the New Hope program in 1998, Aima Marie, a Latina single 
mother of two teenage boys, described this cotmection in her own life. She beheved that the 
program’s benefits motivated her to keep working. Her health insurance costs were about 
S230 per month after leaving New Hope. When her eligibility ended, she had to start budget- 
ing more carefully to accommodate this added expense. She considered working two jobs to 
bring in extra money. Anna Marie said: “I was getting kind of spoiled because I was, like, 
‘The three years are coming up real soon.’ Of course, then I started budgeting ASAP. ‘Here 
comes the time. The three’s up, so start saving. Put a dollar away. Start budgeting again.’” 

During a later visit, she mentioned the psychosocial benefits of the New Hope sup- 
ports for her and her children when the family was enrolled in the program: 

I felt more at ease. I mean . . . you know, to be able to save. I wouldn’t of been 
able to do that. I mean, right now. I’m back to check-to-check. You know, 
back to check-to-check, where I have to say no to my kids sometimes, or my 
kids [say] to me, “We can’t. We can’t afford it because we don’t have the ex- 
tra money right now.” With New Hope, when I did have the extra money, for 
three years [we] had fun. 

The financial support fi'om New Hope helped to improve Aima Marie’s sense of op- 
timism and well-being: “I was a happy person. I looked for that check every month. It made 
me feel so unique, so special.” 
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significantly lower levels of stress but higher levels of time pressure. No impacts were observed 
on depression or self-esteem. 



Five-Year Impacts on Parents’ Well-Being 

Overall, there were few impacts on parents’ well-being. Findings for all variables ap- 
pear in Table 4.1 for the entire survey sample and in Table 4.2 for the survey sample divided by 
employment status at baseline. Findings for breakdowns of the sample by two other baseline 
characteristics — the number of barriers to employment and ethnicity — appear in Appendix E. 
Because of the relatively large number of measures examined, it is possible that some of the 
statistically significant individual impacts may have occurred by chance. Nonetheless, the ob- 
served impacts — especially those reasonably consistent across subgroups defined at baseline 
— seem concentrated in specific domains. 

Material Well-Being 

Measures. Total family income was assessed by asking survey respondents to list all 
income sources for the calendar year preceding the survey, including income from all household 
members. This measure includes sources of income that are not captured by administrative data 
(as were used for the income measures reported in Chapter 3), including income generated by 
paid work, assets, transfers, and miscellaneous other sources. Participants used a five-point scale 
to rate their satisfaction with standard of living. The material hardship index included six ques- 
tions asking whether the family had been without utilities, medical care, housing, or other ne- 
cessities. Food security involved questions about the adequacy of food for the family. Financial 
well-being questions asked about the ability to afford the things the family needs. Financial 
health included nine items assessing participants’ degree of financial stability and ownership of 
such assets as a car, a savings account, or cable TV. Financial support to others included finan- 
cial assistance to family members and others outside the household, other than child support. 
Financial worry included five questions asking how much the respondent worried about paying 
bills and lacking money for important needs such as food and housing. 

Impacts. For the entire survey sample, no impacts were observed on measures of mate- 
rial well-being. For the subset of sample members employed full time at baseline, however, 
program group members were significantly more likely to be providing financial assistance to 
individuals outside their household — other than child support — than were control group 
members. Among those not working full time at baseline and those with two or more employ- 
ment barriers, the situation was the opposite; Program group members were less likely than con- 
trols to be providing such assistance. Among whites, program group members surpassed con- 
trols on the measure of food security. 
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The New Hope Project 
Table 4.1 



Impacts on Parental Well-Being for the Survey Sample 



Outcome 


Range 


Program 

Group 


Control 

Group Difference 


Effect 
P-Value Size“ 


Material well-being 


Family income: survey report ($) 




22,718 


21,270 


1,447 


0.312 0.09 


Standard of living 


l=very unhappy, 
5=very happy 


3.8 


3.7 


0.1 


0.435 0.07 


Material hardship 


0=no, l = yes 


0.2 


0.2 


0.0 


0.664 -0.04 


Food security 


l=low, 4=high 


3.4 


3.4 


0.0 


0.664 -0.04 


Financial well-being 


l=not at all true, 

5=very true 

(Sum of 5 items: 1-25) 


16.2 


16.4 


-0.2 


0.705 -0.03 


Financial health 


0=no, l==yes 


0.5 


0.5 


0.0 


0.716 -0.03 


Financial support to others 


0=no, 100==yes 


9.4 


10.0 


-0.7 


0.799 -0.02 


Financial worry 


l=not at all, 
5=a great deal 


2.5 


2.6 


0.0 


0.672 -0.03 


Residential well-being 


Number of moves in past 3 years 




1.2 


1.2 


0.0 


0.772 0.02 


Satisfaction with housing 


1 =low, 4=high 


3.0 


3.0 


0.0 


0.556 -0.05 


Bad housing conditions 


1 =no, 2==yes 


1.1 


1.1 


0.0 


0.691 -0.03 


Crowded housing 


greater than l=crowded 
housing 


1.5 


1.5 


0.0 


0.780 -0.02 


Neighborhood good to raise kids 


l=awful, 5=excellent 


3.0 


3.0 


0.0 


0.859 -0.01 


Neighborhood resources 


1 =very unhappy, 
5=very happy 


3.5 


3.5 


0.1 


0.354 0.08 


Collective efficacy 


l=very unlikely, 
5=very likely 


3.9 


3.9 


0.0 


0.928 -0.01 


Housing safety: observer 


l=low, 3=high 


1.4 


1.4 


0.0* 


0.073 0.16 


Neighborhood problems: observer 


6=low, 12=high 


8.6 


8.8 


-0.2 


0.525 -0.06 


Physical well-being 


Physical health 


l=low, 5=high 


3.5 


3.4 


0.2 * 


0.058 0.16 


Health condition hinders work (%) 




20.9 


20.7 


0.2 


0.950 0.01 


Hospitalizations in last year 


0=no, l==yes 


0.3 


0.4 


-0.2 


0.270 -0.10 


Substance use/abuse 

Coping strategies and instrumental 
behaviors 


l=on no days, 
5=almost every day 


1.4 


1.3 


0.1 


0.147 0.12 


Sustainability 


l=sort of true for you, 
4=very true 


3.0 


3.0 


0.0 


0.461 0.06 


Pursuing any goals (%) 




80.2 


83.8 


-3.6 


0.287 -0.09 


Achieving any goals (%) 




70.4 


69.1 


1.3 


0.746 0.03 


Awareness of helping resources 


1-low, 2=high 


2.0 


1.9 


0.1 ♦♦ 


0.012 0.22 


Community involvement 


l=low, 2=high 


1.4 


1.3 


0.0 


0.200 0.11 


Married and living with spouse (%) 




23.5 


21.3 


2.1 


0.523 0.05 


Not married and living with partner (%) 




26.1 


28.3 


-2.3 


0.601 -0.05 



(continued) 
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Table 4.1 (continued) 



Program Control Effect 

Outcome Range Group Group Difference P-Value Size^ 

2.6 -0.1 0.538 -0.06 



Discouragement of working 
Practical supports for working 

Used EITC last year (%) 

Aware of EITC last year (%) 

Psychosocial well-being 
General life stress 

Time pressure 
Job quality 

Difficult life circumstances 
Depression 

Religiosity 

Hope 

Influence of close others 
Availability of supportive others 
Perception of life now versus 5 years ago 



l=a big negative influence. 


2.5 


5=a big positive influence 




l=not involved in a 


2.1 


program. 




3=received support 






69.8 




94.3 


l=none of the time. 


2.5 


4=almost all of the time 




l=low, 5=high 


3.3 


0=no benefits. 


0.6 


l=high benefits 




0=no, l=yes 


0.2 


0=low, 60=high 


14.3 


(sum of 20 items) 




l=low, 4=high 


3.7 


l=stongly disagree. 


3.0 


5=strongly aggree 




l=not true at all. 


3.7 


5=always true 




l=not true at all. 


3.1 


5=always true 




l=much worse. 


4.3 


5=much better 





2.0 


0.1 


0.302 


0.09 


67.1 


2.7 


0.514 


0.06 


89.5 


4.8 ** 


0.034 


0.17 


2.5 


0.0 


0.855 


0.02 


3.3 


0.0 


0.706 


-0.03 


0.7 


0.0 


0.208 


-0.11 


0.2 


0.0 


0.414 


0.07 


15.9 


-1.5 ♦ 


0.091 


-0.14 


3.6 


0.1 


0.160 


0.11 


3.0 


0.0 


0.599 


-0.05 


3.6 


0.1 


0.224 


0.11 


3.1 


0.0 


0.687 


-0.04 


4.3 


0.0 


0.906 


-0.01 



Sample size 277 276 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *’*'’*' = 1 percent, *’*' = 5 percent, and ’*' = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 
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General life stress l=none of the time, 2.6 2.6 0.0 0.646 0.05 

4=almost all of the time 

Time pressure l=low, 5=high ^ 3.3 3.3 0.0 0.905 -0.01 
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percent, ft = 5 percent, and t “ 10 percent. 



Residential Weli-Being 

Measures. Residential stability was the number of times the family had moved in the 
previous three years. Other questions tapped parents’ reports of satisfaction with their housing-, 
the extent to which they experienced bad housing conditions, such as malfunctioning plumbing 
or broken windows; crowding; satisfaction with neighborhood resources (such as shopping fa- 
cilities, safety, activities for children); collective efficacy of the neighborhood (neighbors re- 
sponding to crime, helping others); the safety of the housing and neighborhood problems, as 
rated by the survey interviewer. 

Impacts. Of the nine variables examined, there was a significant impact on one: Inter- 
viewers rated housing safety more favorably for program group members than for controls, es- 
pecially for those with two or more employment barriers at baseline. 

Effects on two other indicators of residential well-being varied with baseline employ- 
ment status. Among parents who were employed full time at baseline, program group members 
had moved more often, but were less satisfied with the resources in their neighborhood, than 
were control group members. At the same time, there was a tendency, although not statistically 
significant, for program group members who had not been employed full time at baseline to 
have moved less often than comparable controls, while expressing greater satisfaction with 
neighborhood resources. Afi'ican-American members of the program group likewise expressed 
more satisfaction with neighborhood resources. 

It is unclear whether residential stability is a positive or negative outcome, but, in this 
instance, any improvements would appear to be modest at best, given the absence of significant 
impacts for most of the housing and neighborhood variables. Another indicator of the complex- 
ity of these findings is that, among Hispanics, program group members were more likely than 
control group members to report bad housing conditions and dissatisfaction with their housing. 

Physical Well-Being 

Measures. Survey questions assessed parents’ overall physical health on a five-point 
scale fi-om “poor” to “excellent”; having a health condition that limits the ability to work; fi-e- 
quency of hospitalization during tlie prior year; and indicators of substance use (for example, 
drinking alcohol in the prior 30 days) or abuse (for example, fiiends or family worry that you 
drink too much) . 

Impacts. A single impact on physical well-being was observed for the survey sample as 
a whole. Program group members — especially whites and those not employed at baseline — 
reported better physical health than did controls. This is important for two reasons. First, though 
based on subjective reports, the measure has a long-standing history of use in many national 
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health surveys, and it can be considered a proxy for actual health status. Second, physical well- 
being speaks to the possibility of some “carryover” from New Hope’s assistance with health 
insurance beyond the end of the New Hope offer. In a similar vein, among African-Americans 
and those with one employment barrier at baseline, program group members experienced sig- 
nificantly fewer hospitalizations in the year preceding the survey. 

New Hope had no overall impacts on having a limiting physical condition, frequency of 
hospitalization, or substance use or abuse. At the same time, program group members with one 
employment barrier at baseline displayed a higher level of substance use than did comparable 
control group members. 

Instrumental Behaviors and Coping Strategies 

Measures. Parents’ coping strategies may serve as a bridge for understanding how a 
program that is targeted primarily at adult employment could also have impacts on families and 
children. A key concept, which drew on the ethnographic work, was the sustainability of family 
life, assessed with five items about how capable parents felt in managing their lives, juggling 
responsibilities, and securing resources (see Box 4.2). Parents were also asked if they were pur- 
suing any goals and if they had achieved any goals for themselves. 

Parents were asked about their awareness of helping resources in their community that 
could address such issues as legal assistance or financial counseling, as well as about their own 
community involvement. The parents’ current marital status was determined. Questions about 
whether people or circumstances in their lives discouraged working or provided practical sup- 
port for employment were asked. Parents were asked about their knowledge and use of the 
EITC. 



Impacts. Program group members appear to have had greater awareness of community 
and government resources than control group members. They were significantly more likely to 
know where to turn for assistance with such matters as housing problems, legal issues, financial 
counseling, energy assistance, and child rearing, and they were more familiar with the EITC 
than were controls. Many of these impacts also held for subgroups defined by their baseline 
status — including ethnicity and employment barriers — despite the smaller numbers in these 
subgroups than in the survey sample as a whole. In addition, among those entering the study 
with two or more employment barriers, program group members reported greater community 
involvement than did control group members. Such impacts may have been a useful legacy of 
their interactions with staff and their experiences with benefits at New Hope. 
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Box 4.2 



Family Sustainability Makes Daily Life Manageable 

The concept of family sustainability has evolved over more than three decades of 
ethnographic work with families around the world.* Sustainability captures the emergent 
property familiar to any parent: the achievement of holding an everyday routine of family life 
together. Sustainability in family routines includes juggling social and material resources and 
social support, trying to meet one’s own goals and values, minimizing interpersonal conflict 
and enhancing harmony, and providing stability and predictability in everyday family life. 
Sustainability is helped by the amount of coherence among all these aspects of a family’s 
daily life. It is one pathway through which New Hope might have affected children and par- 
ents positively. 

For example, fieldworkers gave a high sustainability score to Shaquita and John, an 
African-American couple with three young children. The family’s financial situation was 
adequate; Shaquita always paid her bills on time, with John’s help, and she kept a savings ac- 
count for herself and her two older boys. Her work at a local nursing home was steady and 
flexible enough to allow her to take time off when the children were sick and needed to stay 
home fi'om school or daycare. Shaquita enjoyed her job, for the most part, and found working 
to be meaningful. She was proud to be able to pay her own bills and not rely entirely on the 
support of someone else or the government. There was a high level of interpersonal harmony 
and social support in the household. John was helpful — particularly with financial support, 
chores, and child care. Shaquita also had an aunt who could help out with the kids. All the 
children were well behaved, in Shaquita’ s view. Although her work schedule changed fi'om 
time to time, it was set two weeks in advance by her employer, so she could reliably predict 
and manage the balance between routines at work and those at home. 

In contrast, Faye, an African-American single mother of a teenage boy and a toddler, 
had low family sustainability. Although Faye generally scraped enough money together each 
month to pay the rent, her income was extremely unreliable. Her work history was spotty, and 
when she did have a job, she worked for a caterer who could not guarantee work reliably fi'om 
one week to the next. She reported that it was relatively easy to find a low-wage job, but she 
had difficulty finding a job that would give her dependable hours, that was not boring, and 
that fit into the family’s schedule. Faye shared her apartment with several adult members of 
her family, on and off. This situation was highly stressful. Although her relatives occasionally 
helped around the house, Faye was never happy with the results and would have to clean the 
house again anyway. The relatives did help occasionally with household finances, but, more 
often, they relied on Faye for support, living rent-fi'ce in her apartment. Although her brother 
and sister both lived with her for significant portions of the study, they rarely volunteered to 
care for her younger boy. Faye often had to negotiate with them, often promising to exchange 
favors, in order to gain their help. Finally, Faye’s older son was generally well behaved, but 
she worried about his susceptibility to friends who might get into trouble. She also worried 
about his poor school performance, but she did not seem to have enough control over his ac- 
tivities to make sure that he studied when he came home fi'om school. 



*Weisner, 1985, 1996; Weisner, Gibson, Lowe, and Romich, 2002. 
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Indeed, as shown in Chapter 2, only 19 percent of parents still receiving benefits when 
New Hope ended reported that the end of their eligibility was a major problem. Inasmuch as the 
end of New Hope meant the end of eligibility for practical supports such as the wage supple- 
ment and assistance with child care and health insurance, these responses suggest that parents 
were left with something else that tempered the ending of New Hope, such as increased coping 
capacities. It should also be noted that the New Hope Project reached out to members of the 
program group as the end of eligibility approached, to offer advice and assistance with the 
upcoming transition. 

New Hope had no overall impacts on family sustainability or on respondents’ pursuing 
and achieving goals, marital status, supports for employment, or general community involvement. 

Psychosocial Well-Being 

Measures. Survey questions provided measures of the fi'equency with which parents 
had experienced stress in the previous month and their experience of time pressure (that is, the 
extent to which they felt rushed to do the things they needed to do). They were also asked about 
the recent occurrence of 1 7 difficult life circumstances, such as having someone close to them 
die or be imprisoned or being investigated by a Child Protective Services unit. Depression was 
assessed using the Center for Epidemiological Studies-Depression (CES-D)^ scale, a 20-item 
self-report scale tapping the recent fi'equency of depressive symptoms, such as crying or feeling 
lonely. The role of religion in the parents’ lives was assessed with a three-item religiosity index 
tapping such issues as the fi'equency of participation in religious activities and prayer. Parents 
also completed the Hope Scale, a measure of belief in one’s capacity to initiate and sustain ac- 
tions to achieve goals, as well as three scales inspired by the ethnographic work, measuring the 
influence of close others (whether they help and support or demand and stress), the availability 
of supportive others, and job quality, defined in terms of the job’s “family-fnendliness.” Fi- 
nally, parents were asked to rate life now compared with life five years ago using a scale fi'om 1 
(“much worse”) to 5 (“much better”). 

Impacts. Of the ten indicators of psychosocial well-being, program and control group 
members differed on one. Program group members reported a lower fi'equency of depressive 
symptoms than did controls, although absolute levels remained fairly high in both groups. Still, 
the finding is noteworthy because parental depression has negative implications for parenting 
practices and child outcomes.'* 



^Developed by the Center for Epidemiological Studies of the National Institute of Mental Health; see Ap- 
pendix D and Radloff (1977). 

‘*McLoyd, Jayaratne, Cellabo, and Borquez, 1994. 
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For the survey sample as a whole, there were no impacts on frequency of stress, time 
pressure, job quality, difficult life circumstances, religiosity, hope, the influence of close others, 
or the availability of supportive others. The absence of an impact on parents’ religiosity is of 
interest, because, as Chapter 5 shows, there were impacts on children’s involvement with reli- 
gious organizations. 

There were, however, psychosocial impacts on specific subgroups, albeit mostly nega- 
tive. For example, among Hispanics, those in the program group reported less goal pursuit, less 
hope, and more difficult life circumstances. In a similar vein, those with two or more employ- 
ment barriers at baseline reported greater stress and poorer job quality than did controls. How- 
ever, among whites, program group members reported less time pressure than controls. 



Comparison of Results at the Two-Year and Five-Year Follow-Ups 

There were few impacts on material or residential well-being at either the two-year or the 
five-year follow-up. At two years, program group members had fewer unmet medical and dental 
needs, but that difference did not persist at five years, probably because members of the program 
group were no longer more likely than control group members to have health insurance. 

At the two-year follow-up, when New Hope benefits were still available, program 
group members — especially those who had not been employed at baseline — experienced 
greater feelings of time pressure than controls. At the same time, they experienced less stress 
and more feelings of efficacy achieving their goals, perhaps because of the supports that facili- 
tated their efforts. These differences did not persist at five years, but there was a slight indica- 
tion of lasting psychosocial well-being in the lower levels of depressive symptoms among pro- 
gram group members. 

In the five-year follow-up, when program group members had already been without 
New Hope benefits for about two years, a different set of impacts could be expected to emerge. 
Behaviors and strategies to maintain well-being and secure resources, physical health, and a 
slight tendency to psychosocial well-being seemed to predominate in the five-year impacts. In 
addition, levels of stress, time pressure, and depression declined in both the program group and 
the control group between the two- and five-year follow-ups. 



Conclusion 

Overall, the New Hope program had few impacts on the well-being of parents and 
families five years after random assignment and two years after program eligibility ended. In 
considering the well-being of parents across five domains, this chapter shows that impacts were 
concentrated not so much in economic areas (material, financial, housing, neighborhood) but in 
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the ability of program group members to manage various aspects of their own lives and those of 
their families. Program group members were more knowledgeable than controls about how to 
secure various forms of assistance, including the Earned Income Tax Credit. Although scores on 
a measure of depression were fairly high across the survey sample as a whole, they were lower 
among program group members than among controls — an important positive impact. Similar 
psychosocial effects were seen at the two-year follow-up, in terms of increased “agency” and 
hope among program group members, that may have helped to provide a foundation for indi- 
vidual growth and capacity-building. Impacts on physical health two years after the end of New 
Hope eligibility suggest a lasting effect beyond the access to health insurance that was made 
possible by the project. 

Viewing the New Hope Project as the time-limited intervention that it was not intended 
to be, perhaps the program’s overarching benefit was strengthening the capacities and compe- 
tencies of individual participants, thereby enhancing their ability to weather life’s contingencies. 
Any impacts observed in parenting and family management — and ultimately in child out- 
comes — may be a function of an increased capacity among program group members that out- 
lived the cessation of the New Hope Project. This idea would be consistent with a pathway 
largely fi'om instrumental behaviors and coping strategies and psychosocial well-being to par- 
enting and family management and then to child outcomes. This interpretation is also consistent 
with ethnographic evidence regarding parents’ own experiences. Without a doubt, particular 
benefits assisted parents; in addition, however, parents often described positive gains fi'om the 
overall support that they received fiom the New Hope Project. 

While the New Hope Project was not intended to be time-limited, it was intended to of- 
fer an individualized approach for individuals and their families, custom-tailored to their spe- 
cific needs and choices. Differential effects for subgroups of the sample defined at baseline 
seem to indicate that the program did, in fact, function in this way. The intervention offered dif- 
ferent pathways toward different outcomes, and enrollees made their own choices during their 
three-year period of program eligibility. However, New Hope targeted neither parenting prac- 
tices nor children directly, although it did expand the range of possibilities available for child 
care and health care. So a clear and unequivocal pathway to the child outcomes reported in 
Chapter 6 remains somewhat elusive. 



O 
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Chapter 5 



New Hope’s Effects on Parenting, Child Care, 
and Children’s Activities 



Parents’ participation in New Hope could affect children’s well-being through its im- 
pacts on their daily environments, both at home and away from home. This chapter presents 
impacts of the New Hope program on important contexts for children’s development, namely, 
the parent-child relationship, child care arrangements, and out-of-school activities. Two years 
after random assignment, the principal noneconomic impacts of the New Hope intervention 
were on children’s contexts outside the home. Specifically, children from New Hope families 
spent more time in formal child care and structured out-of-school activities than did children 
from control group families. New Hope’s effects on parenting and children’s activities within 
the home (for example, chores) were small or nonexistent at the two-year follow-up. 



Key Findings 

• The New Hope program had few overall effects on parenting and parent-child 
relations; however, parents in the program group reported fewer problems with 
control in discipline situations than did parents in the control group. These dif- 
ferences in parental control were especially strong for children aged 13 to 16. 
There is some evidence that boys in New Hope families had more positive re- 
lations with their parents than did boys in control group families. 

• Children in New Hope families spent more time in formal, center-based care 
and after-school programs, while spending less time in home-based care, 
than children in control group families. Moreover, program group children 
were less likely to spend time in settings that were imsupervised by an adult 
(that is, in self-care or care by someone younger than 16). 

• Adolescent program group children participated in more structured, organ- 
ized activities than did control group adolescents, continuing a pattern begun 
three years earlier. Children of all ages in the program group participated 
more in classes and activities offered by religious institutions. During the 
summer, they spent more time in service or volunteer activities and less time 
attending summer school. 
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Summary of Theoretical Expectations 

Parenting is often defined by the nature of parent-child interactions (for example, affec- 
tion and discipline), but parents also influence their children’s lives by arranging or selecting chil- 
dren’s environments and experiences with others, that is, by being family managers. Family man- 
agement includes selecting child care and arranging for supervision, educational experiences, and 
opportunities for recreation — tasks that become especially critical when parents are employed. 

Parenting 

Income, employment, and parents’ psychological well-being are the principal paths by 
which New Hope was expected to influence parenting and children’s enviromnents. By five 
years after random assignment, the program had very modest lasting effects on each of these 
outcomes, although there were more persistent effects for some subgroups (see Chapters 3 and 
4). Increased income could affect parenting both because it permits parents to provide more ma- 
terial resources to their children and because it may improve parents’ psychological well-being. 
Previous research has demonstrated that low-income parents provide less cognitively stimulat- 
ing home enviromnents, exhibit lower levels of emotional support, less supervision, and more 
punitive discipline than higher-income parents.' These variations may be due in part to differ- 
ences in parents’ psychological well-being and ability to purchase cognitively stimulating mate- 
rials.^ Given the demonstrated links among financial and emotional support, adult psychological 
well-being, and positive parenting,^ the New Hope program was expected to improve parenting 
(for example, through increased expressions of positive affection and more effective family 
management and discipline strategies) by providing families with increased income (including 
income stability) and support services. 

Findings fi"om experimental studies of welfare-to-work demonstrations show very lim- 
ited effects of employment on mothers’ mental health and parenting, perhaps because employ- 
ment can produce both stress and psychological well-being."* It was expected that the benefits of 
employment for program group parents would outweigh the potential stressors created by em- 
ployment because of two salient factors of the New Hope experience. First, employment among 
New Hope participants generated more economic benefits (earnings supplements, health care 
subsidies) than would typically be the case for individuals working in low-wage jobs. Second, 
New Hope staff provided participants with respectful and useful support and assistance. Third, 
the child care subsidy allowed parents to select child care that fit their needs and preferences. 

‘Duncan and Brooks-Gunn, 1997; McLoyd, 1998; Guo and Harris, 2000; Watson, Kirby, Kelleher, and 
Bradley, 1996. 

^Dubow and Ippolito, 1994; Garrett, Ng’andu, and Perron, 1994. 

^Magnuson and Duncan, 2002; McLoyd, 1990. 

‘*Chase-Lansdale and Pittman, 2002. 
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Finally, there were small positive effects on parents’ psychological well-being at both two years 
and five years. 

At the two-year follow-up, there were few effects of New Hope on parenting practices. 
There was some evidence that boys in New Hope families perceived relations with their parents 
as more positive than did boys in the control group. Among parents employed fiill time at ran- 
dom assignment. New Hope significantly increased parental warmth and parent-reported moni- 
toring of the focal child’s activities.^ 

In light of the findings fi’om the two-year follow-up. New Hope was not expected to 
have robust effects on parenting behavior five years after random assignment. Any lasting ef- 
fects were likely to occur for boys or for parents who were employed at baseline, if, for exam- 
ple, New Hope parents perceived that increased warmth and monitoring were promoting posi- 
tive behaviors or were inhibiting problematic behaviors in their children. 

Child Care 

When parents entered the New Hope study, focal children ranged fi-om 1 to 10 years 
old; hence, five years later they were ages 6 to 15. Parental employment, particularly in single- 
parent families, requires child care, at least until children reach late elementary school. The ef- 
fects of parents’ employment on children depend considerably on the kind of care the children 
receive. Low-income parents face particular challenges in arranging care, largely because they 
are less able than middle- and upper-income parents to pay market rates for child care. Conse- 
quently, poor families tend to rely on relatives to care for children.® Although they are less likely 
to use paid care, when low-income parents do pay for care, they spend five times more of their 
income than families who are not poor.’ In fact, child care often represents the second- or third- 
greatest expense for low-income working families.® 

The New Hope program addressed these issues through its child care subsidy.’ Al- 
though public subsidies were also available to some control group parents through Aid to 
Families with Dependent children (AFDC) or through federal and state subsidy programs, the 
income thresholds were considerably lower than those for New Hope, and availability was not 
automatic even for those whose incomes met the eligibility guidelines. The availability of 
child care subsidies can decrease reliance on self-care and informal care by older siblings, 
family, or neighbors and can increase the use of paid, organized child care in centers and or- 



®Huston et al., 2001. 

®Brown-Lyons, Robertson, and Layzer, 2001; Scarr, 1998. 

’Smith, 2000. 

®lsaacs, 2002. 

’chapter 1 presents details of the New Hope child care subsidy and compares it with other pubUc subsidies. 



-93- 

129 



ganized after-school programs.'® Many low-income parents prefer center care and use it when 
it is made available." 

This in fact happened during the first two years of the evaluation: Program group chil- 
dren were more likely than control group children to be in formal settings (such as center-based 
care, preschool programs, and Head Start). Because formal, center-based arrangements typically 
cost more than home-based arrangements, it is reasonable to assume that the New Hope income 
supplement and child care subsidy expanded parents’ child care options. 

Given that families lost the New Hope child care subsidy when the program ended and 
that the impacts on earnings and income had diminished by Year 5, it may be unlikely that New 
Hope would have a lasting effect on the types of child care used by families. On the other hand, 
there are a few reasons why program group families might continue to use more formal care 
than control group families. Initial experience with formal, center-based arrangements might 
lead parents to be familiar and comfortable with securing out-of-home arrangements. Center- 
based care is more reliable for working parents because it is less likely than informal care to fail 
unpredictably.'^ Parents also recognize the cognitive and educational advantages of center- 
based care (see Box 5.2).'® At the same time, parents also believe that home-based care provides 
necessary flexibility to accommodate sick children or employment that is erratic or occurs dur- 
ing nontraditional hours. Parents seek stability and predictability in child care (which is en- 
hanced by flexibility in hours and payments, for instance). 

Children’s Activities 

During the years between third and fifth grades, many children move out of “child 
care,” and parents face new issues of providing supervision and developmental opportunities for 
their children. Out-of-school activities can assume a larger and more important role in chil- 
dren’s development. Structured or organized out-of-school activities and arrangements afford 
the children the opportunity to develop social skills, increase competencies in many domains 
beyond academics, form relationships with caring and supportive adults, and associate with peer 
groups that have similar values and interests.'® They assume particular importance for children 
fi’om low-income families who live in dangerous neighborhoods, because these children are 



'“Lamb, 1997. 

' 'Phillips and Bridgman, 1995; Quint, Polit, Bos, and Cave, 1994. 

'®Hofferth 1999; Lowe and Weisner, Forthcoming, 2003; Lowe, Weisner, and Geis, 2003. 
'®Lowe and Weisner, Forthcoming, 2003. 

'‘*Emlen, Koren, and Schultze, 1999. 

'®Eccles and Barber, 1999; Miller, O’Cormer, and Sirignano, 1995. 
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often considered to be at risk for behavior problems and delinquency if they spend a lot of out- 
of-school time without adult supervision (for example, hanging out with fiiends).'® 

At the two-year point, program group children ages 9 to 12 participated in structured ac- 
tivities (such as organized sports, religious classes and events, clubs, lessons) significantly more 
often than did control group children, possibly because of the greater family income and re- 
sources provided by New Hope. These activities offered adult supervision and organized pro- 
grams. This effect was greater for boys than for girls. 

Although there were not large program impacts on parental income and employment at 
the five-year follow-up point, effects on children’s participation in activities might persist. Chil- 
dren who begin participating in activities in late elementary school may develop interests and 
skills that lead them to continue participation in those, and other, activities. For example, a child 
who plays Little League baseball one year has the skills to join a more advanced team in subse- 
quent years. Children may build lasting adult relationships through participation in organized 
activities. For children at risk, having supportive adults outside the family can offer critical sup- 
port. Through participation in activities, children may also become part of a peer group that 
identifies with the activity. Peers are critically important during the preadolescent period; a 
child’s developmental trajectory may well depend on whether or not he or she becomes part of a 
peer network and, if so, on the type of peer network involved (for example, one that conforms to 
adult norms or deviates fi'om them). Patterns established in preadolescence (roughly ages 9 to 
12) may contribute to resilience and positive developmental trajectories in the adolescent age 
period (ages 13 to 16) that were assessed at the five-year follow-up. 

On the basis of the two-year findings. New Hope’s impacts on children’s participation 
in activities were expected to be greater for boys than for girls. Because of the wide age range in 
the focal child sample, age differences were also expected. Although parents’ baseline employ- 
ment status was an important factor in understanding economic impacts at two years, impacts 
on children’s contexts did not vary systematically for families with different amounts of base- 
line employment; hence, such differences were not expected at five years.’’ 



'^Carnegie Corporation, 1992; Pettit, Bates, and Dodge, 1997; Posner and Vandell, 1994, 1999. 
’’Differences in impacts for families with and without full-time employment at baseline were analyzed for 
all variables, and the differences did not exceed those expected by chance. 
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Impacts on Parenting 



Measures of Parenting 

Measures of parenting included parent reports, child reports, and interviewer ratings. 
Some of these measures were used in the two-year evaluation; they are designated below as 
“(2&5).” Most of the measures were grouped into four composite scores; effective child man- 
agement, positive youth-parent relations, negative youth-parent relations, and warm and struc- 
tured parenting. These groupings were formed partly on the basis of an analysis of all the par- 
enting measures, which indicated that the sets of measures within each grouping were correlated 
with one another.'* These composites were expected to be more reliable than the individual 
scales composing them, because they contained more items. Detailed descriptions of each 
measure, with information about reliabilities, appear in Appendix D. 

Effective Child Management. The composite variable “effective child management” 
represented high control (that is, few problems), mfrequent discipline or punishment, low par- 
enting stress, and high confidence in the ability to prevent harm. Problems with control (2&5) 
was assessed using a five-item scale describing the frequency with which the child ignored or 
failed to obey the parent. ’’ Frequency of discipline involved six items assessing the frequency, 
in the prior week, with which parents had punished the child by grounding, taking away privi- 
leges, and spanking.^" Parenting stress (2&5) included five questions concerning the degree of 
difficulty that parents experienced interacting with and caring for their children.'^' Confidence in 
preventing harm was assessed with a single item from the parent interview: “How confident are 
you that you will be able to prevent your child from getting into trouble?” 

Positive Youth-Parent Relations. The composite “positive youth -parent relations” 
was based on three child report measures; high positive parent-child relations, high parental ac- 
ceptance and involvement, and high monitoring for children age 9 and over. Children’s percep- 
tions of positive relations were assessed by the Child Evaluation of Relationship with 
Mother/Caregiver (2&5).^^ There were different versions for children aged 6 to 8 and for those 
age 9 and older. Both versions included a positive parent-child relations scale. Two additional 
measures were given to 9- to 15-year-olds. One of these was the Authoritative Parenting Meas- 



'*A factor analysis of the measures produced four factors, which accounted for 76 percent of the variation. 
Measures with factor loadings higher than .45 were summed to form four composite scores. Details appear in 
Appendix D. 

'^Statistics Canada, 1995. 

^"Statistics Canada, 1995. 

^'Quint, Bos, and Polit, 1997. 

^^McLoyd, Jayaratne, Ceballo, and Borquez, 1994. 
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ure.^^ The acceptance/involvement subscale assessed the youths’ perceptions that parents were 
supportive and involved in their lives. Children’s reports of parental monitoring were measured 
by asking children ages 9 to 15 about the extent to which their parents knew about their activi- 
ties and their friends. The fact that this scale correlates with other indicators of positive parent- 
child relations (from the child’s point of view) is consistent with recent evidence that “monitor- 
ing” is an index of children’s willingness to communicate with parents rather than of parental 
efforts at supervision.^*' 

Negative Youth-Parent Relations. The composite variable “negative youth-parent re- 
lations” was created for children age 9 and older from two child-report variables; negative par- 
ent relations and low autonomy. The negative relations scale was obtained from the Child 
Evaluation of Relationship with Mother/Caregiver Different versions were given to 

children ages 6 to 8 and to children ages 9 to 15. Children’s perceptions of low autonomy were 
assessed with the subscale psychological autonomy granting, from the Authoritative Parenting 
Measure administered to youth age 9 and older.“ Low scores indicate negative perceptions of 
parents’ willingness to grant the child autonomy. 

Warm and Structured Parenting. “Warm and structured parenting” was composed of 
parents’ reports of warmth, interviewers’ ratings of parental warmth, and parents’ reports of the 
degree to which their family lives were characterized by regular family routines. Parent-reported 
warmth was assessed using a three-item scale from the Canadian evaluation of the Self- 
Sufficiency Project (SSP) (2&5)^’ on which parents indicated the frequency of their praise, fo- 
cused attention, and special activities involving the child. Observed warmth involved two observa- 
tional items from the Home Observation Measure of the Environment (HOME) (2&5). Interview- 
ers assessed whether parents conveyed positive feelings about their children and spontaneously 
praised or talked about their children’s good qualities and behaviors. Regular family routines were 
assessed with six questions about whether there were consistent times for such activities as chil- 
dren’s homework, going to bed on week nights, and eating diimer together as a family. 

Additional Measures of Parenting Behavior. Two parenting measures did not fit in 
the composite groups. One was parent-reported monitoring (2&5), consisting of eight questions 
about parents’ knowledge of their children’s friends and their children’s whereabouts and com- 
panions when away from home. It is worth noting that parents’ and children’s reports of moni- 



^^Steinberg, Lambom, Dombusch, and Darling, 1992. 
^“'Kerr and Stattin, 2000. 

^^McLoyd, Jayaratne, Ceballo, andBorquez, 1994. 
^^Steinberg, Lambom, Dombusch, and Darling, 1992. 
^’Statistics Canada, 1995. 
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toring were only modestly correlated with one another. Parents’ strategies for prevention of 
harm were assessed with six items about the frequency of different strategies, including talking 
to the child, enforcing mles or punishment, involving the child in activities, and keeping the 
child at home. These items were adapted from a set developed by the MacArthur Network on 
Adolescent Development. 

Program Impacts 

Impacts for the Entire Survey Sample. Program impacts on the individual and com- 
posite parenting variables are shown in Table 5.1. There was a significant impact on only one of 
the many measures included: Program group parents reported fewer problems with controlling 
their children than did control group parents. Given the large number of variables tested, this 
finding could have occurred by chance. 

Impacts by Child’s Gender. Impacts on parenting and parent-child relations are pre- 
sented separately by child’s gender in Table 5.2. There was some slight evidence that boys and 
their parents in the program families had more positive relations than did those in the control 
families. In comparison to the control group boys, program group boys reported higher levels of 
parental monitoring, and they were slightly higher (p < .14) on the composite “positive youth- 
parent relations.” (There were no program impacts on parenting for girls.) 

Impacts by Child’s Age. Children were divided into three different age groups based 
on their age as of the five-year survey — 6 to 8, 9 to 12, and 13 to 16 — to determine whether 
program impacts differed across ages. These results are shown in Table 5.3. The striking finding 
from this table is a strong significant program impact on effective child management for the 13- 
to 1 6-year-olds. New Hope parents reported more effective management, including higher lev- 
els of control and less frequent punishment and discipline for their teenage children. There was 
some evidence of a similar pattern for the 9- to 12-year-old children. New Hope parents re- 
ported more confidence in their ability to prevent their children from harm or from getting into 
trouble. There were no impacts on parenting for the youngest age group. 

Impacts by Parents’ Barriers to Employment. In general. New Hope’s effects on 
parenting did not differ according to parents’ barriers to employment at random assignment (see 
Appendix Table E.5.1). Interestingly, program group parents with multiple barriers engaged in 
significantly more strategies to prevent their children from getting into trouble than their control 
group counterparts. 
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The New Hope Project 
Table 5.1 



Impacts on Parenting and Parent-Child Relations for the Survey Sample 



Outcome 


Range 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size^ 


Effective child management 


l=low, 5=high 


3.9 


3.9 


O.l 


0.174 


0.1 1 


Problems with control 


l=never, 

6=all of the time 


2.2 


2.3 


-0.1 * 


0.059 


-0.15 


Frequency of discipline 


l=never, 

4=4 or more times 


1.9 


2.0 


0.0 


0.521 


-0.05 


Parenting stress 


l=not at all true, 
5=very true 


1.7 


1.8 


-0.1 


0.335 


-0.07 


Prevention-of-harm confidence 


l=low, 5=high 


3.7 


3.7 


0.1 


0.421 


0.07 


Positive youth-parent relations 


l=low, 5=high 


4.2 


4.2 


O.l 


0.290 


0.10 


Child-reported positive relations 


l=not at all true, 
5=very true 


4.4 


4.4 


0.0 


0.978 


0.00 


Chi Id -reported acceptance and 
involvement 


l=strongly disagree, 
4=strongly agree 


3.4 


3.4 


0.0 


0.368 


0.08 


Child-reported monitoring 


l=strongly disagree, 
4=strongly agree 


3.2 


3.1 


O.l 


0.222 


0.1 1 


Negative youth-parent relations 


l=low, 5=high 


2.8 


2.8 


0.0 


0.452 


0.07 


Child-reported negative relations 


l=not at all true, 
5=very true 


2.6 


2.5 


0.1 


0.173 


O.IO 


Child-reported autonomy 


l=strongly disagree, 
4=strongly agree 


2.4 


2.5 


0.0 


0.348 


-0.09 


Warm and structured parenting 


l=low, 5=high 


3.9 


3.9 


0.0 


0.399 


0.07 


Parent-reported warmth 


l=never, 

6=many times a day 


4.7 


4.6 


0.1 


0.216 


0.10 


Observer-reported warmth 


l=not at all, 
3=extremely 


2.4 


2.4 


0.0 


0.708 


0.03 


Regularity of family routines 


l=almost never, 
5=almost always 


3.8 


3.8 


0.0 


0.687 


0.04 


Parenting behavior 


Monitoring 


l=never, 6=always 


3.2 


3.2 


0.0 


0.378 


-0.08 


Prevention-of-harm strategies 


l=almost never, 
5=very often 


3.9 


3.8 


0.0 


0.450 


0.06 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = I percent, ** = 5 percent, and * = 10 percent. 

Parent reports were available for 830 children; and child reports were available for 840 children. Actual sample sizes 
for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard deviation 
of the outcomes for both groups combined. This standard deviation is always obtained from the entire research sample, even 
if the table shows impacts for subgroups. 
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Table 5.3 (continued) 






>*s 



o 






0 §■ 

1 I 

c3^ 

B §■ 

S o 

p O 



— m 

o — 



0\ VO 
m cs 
a\ <N 
CO CO 



o — 

CO CO 



CnJ <7\ 
rn 



Cd > r- 

II ^ 5 
VO ^ 



^ ^ 



o 

*ob 



o> 00 A 

•“.So 

Of ^ ‘O 

•S.f § 

goo 

I 2 £ 



Oi 

< 



CN m 
m Tj- 
CO CO 



o r- 
m o 
o o 



<N 

CO 



VO 

CO 



VO 

CO 



OS 

CN 



CN Tj- 
CO CO 



r- vq 
rn (N 



Os ^ 

rn <N 



cp 

(N 



p 

(N 



^ > 



o o 

« u 

CQ C 

II IT 

VO — 



- 1 
Ui « 

15 



T r 

Tf — 



|8 

= Xi 

2 I 



•- Xi 

a s 

g a. 



o 



o 



00 

c 



r- ov 
cn 00 
— o 



<N 

ON 

o 

o 



p 

o 



Ov 

Tj- 

r- 



OO Tj- 

o o 



VO ov 
VO VO 
Kn 



<N 

Tf 

Tf 



— — m 



— <N 
CO CO 



o — ^ 






p 

o 



o o 
o o 



p p 
cn <N <N 



Qo r- 
<N <N 



<N 



X 

op 

u 

S fi 

a cn 



u 

-c p 
op B 
^ zz 

II Cd 
vn ^ 
.. cd 

II 









I ^1 

p ^ 

^ 00 00 00 00 CT) M 

c c c c r. cd 

C o o o o > 

lu i: i: i: i: o o 

> c/1 c/1 c/1 c/1 M C 

II II II II II II II 

^ ^ — 



feb « 

Cd « 

& 

T3 Cd 

p 

■5 00 00 

c c 
Coo 



T3 

cf: 

c 

o 

o 



o 

o. 

T3 

o 

ti 

o 

o. 

o 



T3 

c 

ed 



c 

CL 



t3 

u 

V 

o 

CL 



00 

c 






JC 

u 



« a 

> o 
o V 

> -o 
c ro 

*“ JZ 

u 



c« c 

c o 

O *0 

*•£3 A2 

ed u 
£ > 



' t3 
^ u 
3 ti 

^ qj 

a, 

.2: s 

c8 iS 



B 

o 

c 

o 



ti 

o 

o. 



JZ 

u 



- 105 - 



o 

ERIC 



Warm and structured parenting I=low, 5=high 3.8 3.7 0.0 0.786 0.04 

Parent-reported warmth I==never, 4.5 4.3 o.I 0.406 0.13 

6=many times a day 



Table 5.3 (continued) 
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Impacts by Ethnic Group. Impacts on parenting according to children’s ethnicity are 
presented in Appendix Table E.5.2. The few differences in program impacts across ethnic groups 
suggest that New Hope had some positive effects in this domain for African-American families, 
who reported more effective child management skills, including fewer problems controlling then- 
child. African-American program group parents also reported slightly more regular family rou- 
tines and higher levels of monitoring (both findings were statistically insignificant), and they en- 
gaged in more behaviors to prevent their children from getting into trouble. In contrast, Hispanic 
parents in New Hope reported less effective child management and more problems with control, 
discipline, and parental stress (though program-control differences were statistically insignificant). 
They also reported less regular family routines (statistically insignificant) and less monitoring of 
their children than Hispanic parents in the control group. For white families. New Hope had no 
significant impacts on measures of parenting and parent-child relations. 

Comparison with Two-Year Impacts on Parenting 

The effects of New Hope on parenting after five years were slightly greater than those 
measured at the two-year follow-up. At both times, the impacts were weak and scattered. At the 
two-year follow-up, there were no overall impacts on parenting, but for parents who were em- 
ployed at baseline, there were some indications that New Hope led to increased warmth and 
monitoring.^® At the five-year follow-up, however, employment status at baseline did not mod- 
erate the impact of New Hope, suggesting that this distinction was not crucial as a modifier of 
parenting effects once New Hope ended. The importance of this distinction might well have 
been muted by the varied employment experiences that both program and control group parents 
had over the course of the post-New Hope period. 

At the two-year follow-up, program group boys reported more positive relations with 
their parents than control group boys, and their parents reported fewer problems controlling 
their children. At the five-year follow-up, positive effects continued to be evident for boys and 
not for girls, although the effect sizes were generally small to modest. As in the two-year fol- 
low-up, at five years, program group boys reported slightly more positive relations with their 
parents — a composite variable that reflects higher parental monitoring, acceptance, and in- 
volvement and more positive parent-child relations. , 

New Hope affected parenting behavior differently for children of different ages at the 
five-year follow-up, but not at the two-year follow-up. The five-year follow-up included chil- 



Analyses of five-year impacts for parents employed and not employed at baseline showed only one sig- 
nificant impact, and it was not consistent in direction with the two-year findings. As there were no differences 
in child impacts by employment at baseline, these analyses are not reported here. 



Box 5.1 



Parent-Child Relationships Mediate Adolescents’ 
Adjustments to Increased Parental Employment 



Observations from the New Hope Ethnographic Study (NHES) illustrate the impor- 
tance of parent-child relationships as parents increased employment in response to New 
Hope.* Parents who were working and who had an “understanding” relationship with their 
teenagers — a relationship that included feelings of mutuaUty, trust, and support — tended to 
have children with fewer problem behaviors and conflicts than did working parents who had a 
more problematic relationship with their children. Work requirements and incentives to work 
led mothers to spend less time at home, where they could be directly available to watch and 
monitor their children’s activities. During morning and afternoon and evening “gap times” 
(when there was no school but parents were at work), trust and shared caretaking becomes 
particularly valuable. Families benefit from a kind of “social capital” that comes into play at 
such times. Monitoring indirectly through other children and through negotiated agreements 
between parents and children is common. Adolescents are needed to perform chores and to 
care for siblings, and it is important that they can be trusted to stay home safely alone. With- 
out the kind of trust that is associated with successful parental balancing of work and caretak- 
ing, adolescents and parents negotiate these gap times less well, and youth fare less well. 



*Weisner, Gibson, Lowe, and Romich, 2002. 



dren in mid to late adolescence, a time when issues of monitoring, control, and autonomy are 
especially salient (see Box 5.1). New Hope parents reported more effective management of their 
13- to 16-year-olds than did control group parents, and they expressed more confidence in their 
ability to prevent their children from harm. It is highly plausible that all of the longer-term pro- 
gram effects discussed here reflect, in part, parents’ responses to improvement in children’s be- 
havior over time (for example, higher academic performance, more positive behavior, and less 
problem behavior), stimulated initially by various conditions produced by New Hope (such as 
increased time spent in before/after-school programs, more frequent participation in structured 
activities, and more warm and effective parenting).^’ Box 5.1 describes how the quality of par- 
ent-child relations affect adolescents’ adjustments to increased parental employment. 



^Huston et al., 2001; Lytton, 1990. 
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Impacts on Child Care 



Measures of Child Care 

Parents were asked about the number of months during the prior year that they used 
each of their regular child care arrangements while they were working or away from home. For 
both the school year and the summer, parents were asked whether the focal child had been in 
any formal care (including preschool, before/after-school, community center, or Head Start), 
home-based care by an adult in the family’s home or the caregiver’s home, or care by someone 
16 years old or younger; and whether the child had ever cared for him- or herself or had ever 
provided care for siblings. For analysis purposes, care by a minor, self-care, and caring for sib- 
lings were further categorized as care unsupervised by an adult. 

Parents were asked whether they had paid for any of the child care arrangements they 
had used in the past year and, if so, whether the care had been paid for by themselves, a spouse 
or partner, another family member, or the welfare department or other public agency. Finally, 
parents were asked how much money they (or another family member) had spent on child care 
in the past month for all of their children. 

Program Impacts 

Impacts for the Entire Survey Sample. There were large and consistent program im- 
pacts on the types of child care experienced by the children in the survey sample families. Chil- 
dren from program group families spent significantly more time in center-based care and sig- 
nificantly less time in home-based care than children from control group families. As shown in 
Table 5.4, these program-control group differences in care arrangements occurred both during 
the school year and during the summer months. Program group children also spent less time 
being cared for by a minor during the summer. Across the whole year, program impacts trans- 
lated into an average of one additional month of formal care, one month less of home-based 
care, and approximately two-thirds of a month less of unsupervised care. Despite differences in 
the types of care they used, program and control group families did not differ in their use of paid 
care, the amount they paid for child care out of pocket, or their receipt of public child care assis- 
tance (that is, from an agency or welfare department). 

Impacts by Child’s Gender and Age. Impacts on child care did not differ by the 
child’s gender and are not shown. The impacts by the child’s age at follow-up are shown in Ta- 
ble 5.5. Program effects occurred only for children age 12 and younger, as might be expected. 

For children ages 6 to 8, New Hope families used formal care during the school year 
and during the summer for more months than did control group families. Control group families 
used more home-based care, usually in the child’s home. 
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The New Hope Project 
Table 5.4 



Impacts on Child Care Use and Monthly Child Care Costs for the Survey Sample 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size“ 


During the prior school year, number of months spent in 


Any formal care 


3.0 


2.1 


* 

* 

oo 

o 


0.014 


0.22 


Any home-based care: 


3.7 


4.6 


-0.9 ** 


0.020 


-0.21 


In child's home 


2.4 


3.6 


-1.1 *** 


0.002 


-0.28 


In caregiver's home 


2.0 


2.2 


-0.1 


0.684 


-0.04 


Any unsupervised care: 


2.1 


2.4 


-0.4 


0.235 


-0.10 


Care by a minor 


0.9 


1.1 


-0.2 


0.361 


-0.08 


Self-care 


0.8 


1.2 


-0.3 


0.135 


-0.13 


Cared for sibling(s) 


0.9 


1.1 


-0.2 


0.304 


-0.09 


During the prior summer, number of months spent in 


Any formal care 


1.0 


0.6 


0.3 *** 


0.009 


0.25 


Any home-based care: 


1.4 


1.7 


-0.3 ** 


0.016 


-0.22 


In child's home 


0.9 


1.3 


-0.4 *** 


0.004 


-0.27 


In caregiver's home 


0.7 


0.7 


0.0 


0.826 


-0.02 


Any unsupervised care: 


0.5 


0.8 


-0.2 ** 


0.041 


-0.18 


Care by a minor 


0.2 


0.3 


-0.2 ** 


0.030 


-0.20 


Self-care 


0.3 


0.3 


-0.1 


0.405 


-0.07 


Cared for sibling(s) 


0.3 


0.3 


0.0 


0.669 


-0.04 


During the prior year, number of months spent in 


Any formal care 


3.7 


2.6 


1.1 *** 


0.008 


0.23 


Any home-based care 


4.9 


6.1 


-1.2 *** 


0.008 


-0.24 


Any unsupervised care 


2.6 


3.2 


-0.6 


0.128 


-0.13 


Ever used paid child care during prior year (%) 


42.7 


37.3 


5.4 


0.218 


0.11 


Out-of-pocket child care costs in prior month ($) 


43.4 


37.9 


5.5 


0.573 


0.05 


Received child care subsidy from welfare 


or other agency (%) 


16.4 


16.7 


-0.3 


0.913 


-0.01 


Sample size 


417 


412 









SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 
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The New Hope Project 
Table 5.5 

Impacts on Child Care Use and Monthly Child Care Costs, by Child's Age 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size® 


P-Value for 
Difference 
Across 
Age Groups^ 


Aces 6-8 














During the prior school year, number 














of months spent in 














Any formal care 


3.1 


2.0 


1.1 ** 


0.046 


0.28 


0.863 


Any home-based care: 


3.7 


4.8 


- 1.0 * 


0.070 


- 0.24 


0.673 


In child’s home 


2.2 


3.4 


- 1.2 ** 


0.032 


- 0.30 


0.790 


In caregiver's home 


2.6 


3.1 


- 0.5 


0.329 


- 0.15 


0.532 


Any unsupervised care: 


1.4 


1.2 


0.3 


0.504 


0.07 


0.016 tt 


Care by a minor 


1.1 


0.8 


0.3 


0.299 


0.12 


0.043 tt 


Self-care 


0.2 


0.3 


- 0.1 


0.371 


- 0.05 


0.327 


Cared for sibling(s) 


0.4 


0.5 


0.0 


0.964 


0.00 


0.100 t 


During the prior summer, number of 














months spent in 














Any formal care 


1.0 


0.5 


0.5 ** 


0.012 


0.38 


0.589 


Any home-based care: 


1.4 


1.6 


- 0.2 


0.247 


- 0.15 


0.790 


In child’s home 


0.8 


1.1 


- 0.4 * 


0.050 


- 0.26 


0.717 


In caregiver's home 


0.9 


0.8 


0.0 


0.810 


0.04 


0.753 


Any un supervised care: 


0.4 


0.4 


- 0.1 


0.665 


- 0.04 


0.356 


Care by a minor 


0.2 


0.2 


0.0 


0.890 


- 0.02 


0.204 


Self-care 


0.1 


0.1 


0.0 


0.802 


- 0.02 


0.784 


Cared for sibling(s) 


0.1 


0.1 


0.0 


0.983 


0.00 


0.393 


During the prior year, number of 














months spent in 














Any formal care 


5.1 


3.6 


1.4 ** 


0.025 


0.31 


0.383 


Any home-based care 


5.0 


6.3 


- 1.3 * 


0.059 


- 0.25 


0.694 


Any unsupervised care 


1.7 


1.6 


0.2 


0.737 


0.03 


0.024 tt 


Out-of-pocket child care costs in prior 














month ($) 


63.8 


69.4 


- 5.5 


0.780 


- 0.05 


0.194 


Sample size 


139 


141 










Aces 9-12 














During the prior school year, number 














of months spent in 














Any formal care 


1.8 


0.8 


1.0 * 


0.074 


0.25 




Any home-based care: 


4.1 


4.8 


- 0.8 


0.178 


- 0.18 




In child's home 


2.6 


3.7 


- 1.2 ** 


0.037 


- 0.29 




In caregiver's home 


2.0 


1.9 


0.1 


0.871 


0.02 




Any unsupervised care: 


1.7 


3.0 


- 1.3 


0.004 


- 0.36 





(continued) 
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Table 5.5 (continued) 















P-Value for 
Difference 




Program 


Control 






Effect 


Across 


Outcome 


Group 


Group 


Difference 


P-Value 


Size® 


Age Groups^ 


Care by a minor 


0.8 


1.6 


-0.8 ** 


0.016 


-0.31 




Self-care 


0.8 


1.5 


-0.7 * 


0.055 


-0.25 




Cared for sibling(s) 


0.5 


1.3 


-0.8 ** 


0.020 


-0.28 




During the prior summer, 
number of months spent in 














Any formal care 


0.7 


0.4 


0.2 


0.172 


0.18 




Any home-based care: 


1.4 


1.8 


-0.4 ** 


0.046 


-0.26 




In child's home 


1.0 


1.4 


-0.4 ** 


0.030 


-0.30 




In caregiver's home 


0.6 


0.7 


-0.1 


0.434 


-0.10 




Any unsupervised care: 


0.5 


0.8 


-0.3 ** 


0.047 


-0.26 




Care by a minor 


0.2 


0.5 


-0.3 ** 


0.025 


-0.34 




Self-care 


0.2 


0.3 


-0.1 


0.440 


-0.09 




Cared for sibling(s) 


0.2 


0.3 


-0.1 


0.240 


-0.16 




During the prior year, number of 
months spent in 














Any formal care 


3.7 


2.6 


1.1 * 


0.076 


0.24 




Any home-based care 


5.3 


6.6 


-1.2 * 


0.071 


-0.24 




Any unsupervised care 


2.2 


3.8 


-1.7 *** 


0.003 


-0.36 




Out-of-pocket child care costs in 
prior month ($) 


45.6 


24.3 


21.3 * 


0.089 


0.18 




Sample size 


136 


159 










Aees 13-16 














During the prior school year, number 
of months spent in 














Any formal care 


1.4 


1.2 


0.3 


0.565 


0.07 




Any home-based care: 


3.3 


3.7 


-0.3 


0.579 


-0.08 




In child's home 


2.6 


3.4 


-0.7 


0.233 


-0.18 




In caregiver's home 


1.5 


1.3 


0.2 


0.713 


0.05 




Any unsupervised care: 


3.4 


3.3 


0.0 


0.964 


0.01 




Care by a minor 


0.8 


1.0 


-0.2 


0.516 


-0.09 




Self-care 


1.8 


2.0 


-0.2 


0.747 


-0.07 




Cared for sibling(s) 


1.9 


1.7 


0.2 


0.700 


0.08 




During the prior summer, number of 
months spent in 














Any formal care 


0.6 


0.5 


0.1 


0.423 


0.12 




Any home-based care: 


1.3 


1.6 


-0.2 


0.269 


-0.17 




In child's home 


1.1 


1.3 


-0.2 


0.331 


-0.15 
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Table 5.5 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size" 


P-Value for 
Difference 
Across 
Age Groups*’ 


In caregiver's home 


0.6 


0.6 


0.0 


0.908 


-0.01 




Any unsupervised care: 


0.9 


1.1 


-0.2 


0.294 


-0.18 




Care by a minor 


0.1 


0.2 


-0.1 


0.181 


-0.16 




Self-care 


0.7 


0.8 


-0.1 


0.612 


-0.11 




Cared for sibling(s) 

During the prior year, number of 
months spent in: 


0.5 


0.4 


0.1 


0.586 


0.10 




Any formal care 


1.9 


1.5 


0.4 


0.424 


0.09 




Any home-based care: 


4.5 


5.1 


-0.5 


0.487 


-0.10 




Any unsupervised care: 


4.2 


4.3 


-0.1 


0.931 


-0.01 




Out-of-pocket child care costs in prior 


month ($) 


13.4 


18.1 


-4.7 


0.642 


-0.04 




Sample size 


131 


107 











SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 
Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the 
standard deviation of the outcomes for both groups combined. This standard deviation is always obtained from 
the entire research sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup 
dimensions featured in this table. This p-value represents the probability that apparent variation in impacts across 
each of these dimensions is simply the result of random chance. If this probability is less than 10 percent, the 
variation in impacts is considered statistically significant. Statistical significance levels are indicated as ttt = 1 
percent, tt = 5 percent, and t = 10 percent. 
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For children ages 9 to 12 — the years when most children discontinue formal child care 
— the major program impacts occurred for unsupervised care. Program group children spent 
less time than control group children in unsupervised care, including care by a minor, self-care, 
or caretaking responsibility for other children. Over the entire year, they spent less time in 
home-based care, less time in any unsupervised care, and more time in formal care. 

Impacts by Parents’ Barriers to Employment. There were few differences in impacts 
on child care according to parents’ barriers to employment at random assignment (see Appendix 
Table E.5.3). The impact of New Hope on the use of formal care during the school year was 
concentrated in the group of parents who had no initial barriers to employment. In addition, 
program group parents in this “no barriers” group reported paying significantly more for child 
care than control group parents did. 

Impacts by Ethnic Group. New Hope led to increased use of formal care for children 
from African-American, Hispanic, and white families; however, there were some differences 
among these groups in program effects on other types of care. The negative program impacts on 
families’ use of home-based and unsupervised care were strongest for children in white fami- 
lies. In contrast, for children in Hispanic families. New Hope led to slightly more time in unsu- 
pervised settings (though program-control group differences in this subgroup were not statisti- 
cally significant). These subgroup results are presented in Appendix Table E.5.4. 

Comparison with Two-Year Impacts on Child Care 

Despite the fact that program group families did not have access to New Hope’s child 
care assistance after Year 3, they continued to use more formal care than control group families, 
as they had during the first two years of the program. Moreover, the five-year findings suggest 
that the increased use of formal arrangements by program group families translated into reduc- 
tions in the use of home-based care and of care that was unsupervised by an adult. 

New Hope’s two-year impacts on formal care may be attributed to its child care subsidy, 
which made this type of arrangement more affordable and more accessible. Why would families 
continue to use different types of care two years after the end of the program (and five years after 
random assignment), especially given the absence of sizable long-term effects on earnings and 
income? Perhaps, with initial experience in securing formal, center-based arrangements for their 
children, parents were more familiar with and better able to continue using such arrangements in 
the future. Qualitative data from the New Hope Ethnographic Study suggest that low-income par- 
ents like the stability and predictability of formal care when used for employment purposes.^® In- 
deed, the use of formal care may be related to New Hope parents’ greater stability of employment. 



^°Lowe and Weisner, 2001. 
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Stable employment makes it possible to sustain center-based care; the reliability of such care may 
also have contributed to parents’ ability to maintain stable employment. 

Parents may have perceived some positive effects of formal care for their children. In eth- 
nographic interviews, parents said they thought that formal care contributes to children’s academic 
skills. Recent research indicates that children who attend child care centers in the infant and pre- 
school years perform better on cognitive and language tasks and show better school achievement 
than do those who spend time in home-based care of comparable quality.^’ The positive effects of 
center-based care endure into the first few years of school.^^ The two-year impacts of New Hope 
on children’s academic functioning may, in part, reflect benefits of increased time in center-based 
care. Box 5.2 provides ethnographic evidence that some parents worked to maintain the same type 
of child care arrangements that they had established during New Hope. 

Finally, as mentioned in Chapter 2, New Hope project representatives worked to help 
families adjust to the loss of New Hope benefits at the end of the three-year program. Project rep- 
resentatives may have increased participants’ awareness of other resources for child care assis- 
tance (to replace the New Hope child care subsidy) and may have helped them transition to the 
W-2 subsidy system. This may have contributed to the sustained positive effect of New Hope on 
the use of formal child care. However, it is noteworthy that by two years after New Hope’s end, 
program group families were not more likely to be receiving any public child care assistance. 
Given that program families were not spending significantly more money out of pocket on child 
care than control group families were — despite their increased use of formal care — it may be 
that they were able to negotiate or access resources for child care outside the W-2 system. 



Impacts on Children’s Activities 

Measures of Children’s Activities 

Parents reported on all focal children’s use of time and on their participation in out-of- 
school activities, and children ages 9 to 16 also provided self-reports. Respondents were asked 
first about participation in activities during the school year and then about activities during the 
summer. With the exception of television viewing, responses for all questions were assessed 
using a five-point scale ranging fi'om “never” to “about every day.” 

Structured Activities. Respondents reported how fi'equently children participated in 
lessons, organized sports, clubs and youth groups, and religious classes and events and how 



^'NICHD Early Child Care Research Network, 2000; Zaslow, McGroder, Cave, and Mariner, 1999. 
^^Yoshikawa ,1999; Broberg, Wessels, Lamb, and Hwang, 1997. 



Box 5.2 



Parents Who Put Their Young Children in Child Care 
or After-School Programs Generally Had Positive Experiences 
That Carried Over After New Hope’s Benefits Ended 

Parents hold strong opinions about child care, and those beUefs shape their child care 
decisions. Some families in the New Hope Ethnographic Study (NHES) sample described 
profound distmst of strangers working in child care centers or in family child care homes. 
Fearing that their young children would be neglected or abused physically or sexually, they 
cobbled together child care from their friends and relatives or chose to stay at home to care for 
their children themselves. They pieced together financial support from boyfiiends, spouses, 
fiiends, relatives, or the state. Other parents beUeved that the home environment they could 
provide their children was neither socially nor academically stimulating enough, and they 
sought child care opportunities that would supplement home care. Many parents found it best 
to blend home care and some degree of center care. 

When need and opportunity in the form of financial supports induced parents to try 
care options that they previously had felt were undesirable (particularly child care centers), 
they often formed positive judgments about the impact on their children. Evelia, a Puerto Ri- 
can single mother of four in the control group, worked an eight-hour shift beginning at 3 P.M. 
She did not tmst that her 3 -year-old daughter, Lisa, would be safe in a daycare center, and she 
also doubted that she could find a center that would cover her late work hours (a well-founded 
belief), so she left Lisa with her three older children — the oldest of whom was 13 — or with 
nearby relatives. Eventually Evelia started working the early shift. Partly because her relatives 
were unable to provide reliable care at that time, she applied for child care subsidies from 
Wisconsin Works (W-2) and — despite her misgivings — chose a center for Lisa. After a few 
months, Evelia was thrilled with the center. She thought that Lisa had learned a lot in a short 
amount of time, including lessons that Evelia could not have taught at home. 

Families in New Hope had similar experiences, in which a combination of work, 
subsidies, and support from New Hope representatives encouraged them to enroll their 
children in formal child care and to sustain these arrangements over time. When these 
families’ eligibility for New Hope benefits ended, they effectively negotiated supports to keep 
their children in the care settings they preferred. Andrea, a single Afiican-American mother of 
a preschool-age daughter, realized in 1995 that she could not remain on AFDC to care for her 
daughter indefinitely and would have to find work to support the family. With her brother’s 
help, she found a job as a housekeeper at the community center, learned about and signed up 
for New Hope, and used the program’s child care subsidy to place her daughter in full-time 
child care. She maintained these arrangements for the three years of her New Hope eligibility. 
When that ended, she was concerned about finding a child care arrangement that would 
continue the same kind of program and that she could afford. Fortunately, Andrea was able to 
enroll her daughter in a local school program that covered most of her child care needs, with 
the exception of a short time in the morning and in the afternoon. She arranged to have these 
periods covered by the former child care center that her daughter had attended since infancy, 
and the school program provided bus service to and from the center. 
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often they attended recreation or community centers. These five activities were classified as 
“structured activities” at the two-year evaluation, because they afforded opportunities for adult 
supervision, the acquisition of skills, and socializing with peers. A summary score for stmctured 
activities was computed from the average of these five activities. 

In addition, respondents were asked about participation in programs to help with 
schoolwork, before/after-school programs, daycamp, and summer school. 

Service and Work for Pay. One question asked about children’s participation in ser- 
vice or volunteer activities. Another question asked about work for pay away from home. These 
questions were asked only about children ages 9 and older. 

Parent Approval. Parents were asked about the extent to which they considered par- 
ticipation in five activities to be beneficial or harmful for their child, using a five-point scale. 
The activities were lessons, sports with a coach, clubs and youth groups, recreation centers, and 
working for pay. The average of their answers was the score for parent approval. 

Social Activities. One question asked how often children went shopping or out to eat 
with an adult family member. Another asked about hanging out with friends without an adult. 

Media Use. Questions about the frequency of television viewing at three different times 
of day on weekdays and weekends were asked for the school year. The response scale was five 
points, ranging from “none” to “three or more hours” per day. 

Program Impacts 

Impacts for the Entire Survey Sample. As shown in Table 5.6, program group par- 
ents reported that their children engaged in significantly more religious classes and events than 
control group children, both during the school year and during the summer. Program group 
children spent less time in daycamp or summer school. 

Parents in the program group reported that their children spent more time in service or 
volunteer activities during the summer. During the school year, program group children worked 
for pay more than control group children did, although this impact was only marginally signifi- 
cant (p = .12). 

There were small and inconsistent effects on children’s social activities. Parents in both 
groups reported less shopping and eating out by their children, but children in New Hope fami- 
lies reported more such activities than control group children. There were no overall effects on 
media use. 
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The New Hope Project 
Table 5.6 



Impacts on Childrens's Activities for the Survey Sample 



Outcome 


Range 


Program 

Group 


Control 

Group 


Difference 


P- Value 


Effect 

Size® 


Structured activities - school year 
Total structured activities: parent report 


1 =mever, 

5=about every day 


2.4 


2.3 


0.1 


0.218 


0.10 


Lessons 




2.1 


2.2 


-0.1 


0.160 


-0.11 


Sports with a coach 




2.3 


2.3 


0.1 


0.425 


0.06 


Club/youth group 




2.4 


2.2 


0.2 


0.199 


0.10 


Religious class or activity 




2.9 


2.7 


0.2 ♦♦ 


0.028 


0.18 


Community center 




2.4 


2.3 


0.1 


0.238 


0.10 


School-related programs: parent report 


Program to help with school 




2.3 


2.3 


0.0 


0.705 


0.03 


Before/after-school program 




2.2 


2.0 


0.2 ♦ 


0.098 


0.14 


Total structured activities: child report 




2.6 


2.6 


0.1 


0.532 


0.06 


Lessons 




2.5 


2.5 


0.0 


0.767 


0.03 


Sports with coach 




2.7 


2.7 


0.0 


0.906 


0.01 


Club/youth group 




2.3 


2.3 


0.0 


0.864 


0.02 


Religious class or activity 




3.1 


2.9 


0.1 


0.325 


0.10 


Community center 




2.5 


2.4 


0.0 


0.784 


0.03 


School-related programs: child report 


Program to help with school 




2.0 


2.1 


-0.1 


0.480 


-0.07 


Before/after-school program 




1.6 


1.5 


0.1 


0.357 


0.09 


Structured activities - summer 
Total structured activities: parent report 


l==never, 

5=about every day 


2.2 


2.2 


0.1 


0.368 


0.08 


Lessons 




1.7 


1.8 


0.0 


0.903 


-0.01 


Sports with a coach 




2.0 


2.1 


-0.1 


0.632 


-0.04 


Club/youth group 




2.2 


2.1 


0.1 


0.539 


0.05 


Religious class or activity 




2.9 


2.5 


0.3 ♦♦ 


0.010 


0.22 


Community center 




2.4 


2.3 


0.0 


0.758 


0.03 


School-related programs: parent report 


Summer school 




2.1 


2.3 


-0.2 


0.247 


-0.10 


Daycamp 




1.7 


1.9 


-0.2 ♦ 


0.080 


-0.15 


Total structured activities: child report 




2.2 


2.1 


0.1 


0.307 


0.10 


Lessons 




.1.6 


1.6 


0.0 


0.814 


-0.02 


Sports with a coach 




2.2 


2.2 


0.0 


0.840 


0.02 


Club/youth group 




1.9 


1.8 


0.2 


0.200 


0.12 


Religious class or activity 




3.0 


2.7 


0.3 ♦♦ 


0.021 


0.22 


Community center 




2.3 


2.3 


-0.1 


0.625 


-0.05 


School-related programs: child report 


Summer school 




1.9 


2.3 


-0.4 ♦♦ 


0.019 


-0.22 


Daycamp 




1.7 


1.7 


0.0 


0.913 


-0.01 



(continued) 
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Table 5.6 (continued) 







Program 


Control 






Effect 


Outcome 


Range 


Group 


Group 


Difference 


P-Value 


Size^ 


Service and work for pay - school year 


1 =never, 

5=about every day 












Service and volunteer: parent report 




1.7 


1.6 


0.1 


0.263 


0.08 


Service and volunteer: child report 




1.8 


1.9 


-0.1 


0.737 


-0.04 


Work for pay: parent report 




1.5 


1.4 


0.1 


0.120 


0.11 


Work for pay: child report 




2.0 


1.8 


0.2 


0.203 


0.15 


Service and work for pay - summer 














Service and volunteer: parent report 




1.7 


1.4 


0.3 *** 


0.001 


0.27 


Service and volunteer: child report 




1.6 


1.7 


-0.1 


0.697 


-0.05 


Work for pay: parent report 




1.4 


1.4 


0.1 


0.235 


0.09 


Work for pay: child report 




2.2 


2.1 


0.1 


0.523 


0.08 


Parent approval of participation 


l=not at all, 
5=a lot 


2.9 


2.8 


0.1 


0.440 


0.06 


Social activities - school year 


l=never, 

5=about every day 












Shop or eat out with adult: parent report 




3.3 


3.4 


0.0 


0.528 


-0.05 


Hang out with friends: parent report 




2.8 


2.9 


-0.1 


0.447 


-0.06 


Shop or eat out with adult: child report 




3.4 


3.4 


0.0 


0.600 


0.05 


Hang out with friends: child report 




3.8. 


3.7 


0.0 


0.907 


0.01 


Social activities - summer 


1 =never, 

5=about every day 












Shop or eat out with adult: parent report 




3.3 


3.5 


-0.1 ** 


0.042 


-0.17 


Hang out with friends: parent report 




2.9 


2.9 


0.0 


0.984 


0.00 


Shop or eat out with adult: child report 




3.5 


3.3 


0.2 * 


0.051 


0.18 


Shop or eat out with adult: child report 




3.9 


3.7 


0.1 


0.390 


0.08 


Television - school year 


l=none, 

5=three or more hours 












Watch TV (hours/day): parent report 




2.7 


2.7 


0.0 


0.380 


-0.07 


Watch TV (hours/day): child report 




3.1 


3.2 


-0.1 


0.311 


-0.10 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Parent reports were available for 830 children; and child reports were available for 840 children. Actual sample 
sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 
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Impacts by Child’s Gender. Most of the impacts (or lack of impacts) on children’s ac- 



tivities were similar for boys and girls; however, as shown in Table 5.7, there were a few differ- 
ences by gender. Program group boys reported more time shopping and eating out, and less televi- 
sion viewing, than boys in control group families. For girls, youth and parent reports generated 
conflicting information about participation in service activities; New Hope parents reported more 
participation during the summer — but their daughters reported less participation — than did par- 
ents in control group families. Similarly, New Hope girls reported less shopping and eating out 
with an adult than did control group girls. There were no program effects on girls’ media use. 

Daggers in the column labeled “P-Value for Difference Between Boys and Girls” indi- 
cate that the program impacts on boys were significantly different than the impacts on girls for a 
number of the youth-reported variables, but not for the parent-reported variables. For total par- 
ticipation in structured activities, boys’ reports indicated slightly positive program impacts, but 
girls’ reports indicated slightly negative impacts. Similarly, boys’ reports indicated more posi- 
tive impacts on participation in service activities than did girls’ reports. 

Impacts by Child’s Age. Given the wide age range of the children, different patterns of 
participation in activities were expected for different age groups. The pattern of impacts across 
ages was similar in some respects, as shown in Table 5.8. In particular, the pattern of higher par- 
ticipation in religious classes and related activities appeared across all three age groups, al- 
though it was more pronounced for adolescents aged 13 to 16 than for the two groups of 
younger children. Moreover, program youth ages 13 to 16 participated more frequently in stmc- 
tured activities overall than control group youth. The oldest group was also the only one in 
which program group parents were more likely than control group parents to approve of their 
children’s participation in stmctured activities. 

The youngest program group children (ages 6 to 8) were more likely than control group 
children to attend before/after-school programs as well as summer school, perhaps because their 
parents used these programs as child care. 

The program group children ages 9 and older were less likely than control group chil- 
dren to attend summer school, possibly because they were less likely to be performing poorly in 
school. Children over age 9 in the program group were more likely than their control group 
counterparts to participate in service activities, at least according to parent reports. Youth re- 
ports among the younger children indicated less participation in volunteer activities by the pro- 
gram group than the control youth (Table 5.8). Surprisingly, New Hope’s impact on work for 
pay was greater for children ages 9 to 12 than for the older children. 

Program group children ages 9 to 12 spent less time shopping and eating out with an 
adult than control group children, and they spent correspondingly more time hanging out with 
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Impacts on Children's Activities, by Gender 
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5=about every day 

Total structured activities: parent report 2.2 2.2 0.0 0.725 0.04 0.700 

Lessons 1.6 1.7 -0.1 0.299 -0.10 0.212 

Sports with a coach 2J 2.2 -0.1 0.523 -0.07 0.768 



Table 5.7 (continued) 
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Community center 2.4 2.2 0,2 0.383 0,10 



Table 5.7 (continued) 
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Shop or eat out with adult: parent report 3.4 3.5 -0.1 0.125 -0.17 

Hang out with friends: parent report 2.7 2.7 0.0 0.878 -0.02 

Shop or eat out with adult: child report 3.5 3.5 0.0 0.756 0.04 

Hang out with friends: child report 3^ 3.5 p i 0.567 0.08 
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Table 5.8 

Impacts on Children's Activities, by Age 
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Parent approval of participation l=notatall, 2.7 2.9 -0.2 

5=a lot 



Table 5.8 (continued) 
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Betore/atter-school program 2.1 2.0 0.0 0.834 0.03 

Total structured activities; child report 2.6 2.6 0.1 0.508 0.08 

Lessons 2.5 2.5 0.1 0.717 0.04 

Sports with a coach 2.7 2.6 0.1 0.479 0.08 



Table 5.8 (continued) 
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5=about every day 

Shop or eat out with adult: parent report 3.3 3.5 -0.2 * 0.080 -0.22 

Hang out with friends: parent report 3.0 3. 1 -0. 1 0.5 1 3 -0.08 

Shop or eat out with adult: child report 3.4 3.3 0. 1 0.259 0. 1 3 

Hang out with friends: child report T8 3.4 0.4 * 0.050 0.26 
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Table 5.8 (continued) 
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peers without an adult during the summer. The pattern was reversed for the older children: Pro- 
gram group youth spent less time hanging out with peers. Box 5.3 describes some parents’ con- 
cerns about the potentially negative influences of adolescent peer groups. 

Impacts by Parents’ Barriers to Employment. For the most part, impacts did not differ 
across groups of children whose parents had different numbers of employment barriers; however, 
there were a couple of differences. Program group children whose parents had two or more poten- 
tial barriers participated more in clubs/youth groups than control group children, and those whose 
parents had one potential barrier participated less in lessons (see Appendix Table E.5.5). 

Impacts by Ethnic Group. On the whole, impacts were similar for children of differ- 
ent racial and ethnic groups. There were a few exceptions: Program group children in African- 
American families participated less in lessons, while program group children in white families 
worked less for pay than did control group children (see Appendix Table E.5.6). 



Conclusion 

At the five-year follow-up. New Hope’s impacts on children’s contexts outside the fam- 
ily were considerably stronger than its impacts on parenting and parent-child relations. The pro- 
gram had strong, lasting effects on child care. Despite the fact that New Hope subsidies did not 
continue into the year before the assessment, program group parents used more formal, center- 
based and after-school care and less home-based care than did control group parents, especially 
for their early elementary-school-age children. Their preadolescent children (ages 9 to 12) spent 
less time in unsupervised care, particularly during the summer. Given the evidence that center- 
based care and after-school programs can positively impact children’s intellectual and cognitive 
development, these experiences may have contributed to the better academic performance of 
New Hope children at two years and at five years (see Chapter 6).^^ 

For children in late childhood and adolescence, involvement in structured and organ- 
ized activities (religious, service, or volunteer) can offer advantages for positive development 
and protection from risk. Adolescent program group children participated in more structured 
activities, continuing a pattern begun three years earlier. Children of all ages in the program 
group participated more in classes and activities offered by religious institutions. During the 
summer, they spent more time in service or volunteer activities. Engaging in structured and 
civic activities is linked to positive psychosocial and academic outcomes for children in general, 
and it may be especially pertinent for low-income children, because they are often at risk for 
deviant behavior. Involvement in important institutions such as school and church may provide 



^^Broberg, Wessels, Lamb, and Hwang, 1997; NICHD Early Child Care Research Network, 2000; Yoshi- 
kawa, 1999; Zaslow, McGroder, Moore, and LeMenestral, 1999. 



Box 5.3 



Parents of Adolescents Shared a Concern About 
Negative Peer Influences and Tried to Arrange Opportunities 
for Positive, Adult-Mentored Peer Activities 

Although parents were sometimes fearful about adult strangers caring for their young 
children, their concerns for adolescent children shifted to the risks associated with the unsu- 
pervised peer group. One reason offered by several parents for eiu'olling their children in 
adult-mentored activities at local teen centers and in religious youth groups was to help pro- 
tect their adolescents from the drugs, gangs, violence, and premature sexual activities that 
could put them on the wrong path into adulthood and instead provide opportunities for experi- 
ences with positive adult role models, enjoyable activities, and adult supervision. 

Lisa, an African-American mother of three in the New Hope group, enrolled all three 
of her teenage sons in a local religious center for teens, which offered the boys assistance, tu- 
toring, and a place to hang out with friends — in addition to regular studies of the Bible. Lisa 
encouraged her boys’ participation in this center partly because she felt that her own involve- 
ment in a religious center as a teen had helped her to stay out of trouble. She was particularly 
pleased that her boys enjoyed participating in this organization. Her oldest son, Aaron, would 
even occasionally recmit some of his peers from the neighborhood to attend the center, be- 
cause he felt .it had made such a positive contribution in his life. Most of the boys’ friends 
came from the center, and Lisa felt that none of them would be a “bad influence” on her boys, 
because none of them were involved with gangs or drugs. The center also gave her sons valu- 
able leadership opportunities. For example, each boy participated in coaching a summer bas- 
ketball league for Wds. 

In deciding whether or not to enroll their teenage children in community-based pro- 
gram for youth, several parents were clearly attempting to shape the kinds of peers to whom 
their sons or daughters might be exposed. Generally, parents were wary about programs that 
attracted what one parent called “the rough kids.” Maria, a married Latina mo&er of four in 
the control group, refused to enroll her children in a local YMCA program for youth because 
she was concerned about the kinds of kids that participated in it — who she thought might be 
drawn from gangs, for example, and would be a negative influence on her children. She did 
enroll her oldest son, a teenager, in a local religious program that was designed to keep kids 
away from gangs and drugs. He attended the program every Saturday morning and seemed to 
enjoy it. Maria thought that the kids in this program would be a good influence in her son’s 
life and that the lessons would protect him from the hazards that kids faced in the neighbor- 
hood and at school. 
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connections to supportive adults and prosocial peers that sustain a positive developmental tra- 
jectory. These generally positive impacts on children’s participation in activities coincided with 
school engagement, educational expectations, feelings of efficacy, and positive social behavior 
(see Chapter 6). 

By contrast. New Hope had relatively small impacts on parenting at both the two- and 
five-year follow-ups, but there were some trends for parents of program group adolescents to 
feel more in control and for boys and their parents to have more positive relationships. The 
slight increase in program impacts from two to five years could be a result of parents’ responses 
to changes in children’s behavior as well as direct effects of the program on parenting. 

The findings in Chapter 6 show that the program impacts on children were stronger and 
more positive for boys than for girls. Although impacts on both parenting and out-of-school 
contexts were slightly more positive for boys than for girls, these differences do not appear to be 
sufficiently large to account for the different impacts on child outcomes. Finally, there were few 
consistent differences across impacts on parenting, child care, and children’s activities accord- 
ing to parents’ initial employment barriers or ethnicity. 
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Chapter 6 

New Hope’s Effects on Children 



This chapter examines New Hope’s outcomes for children and youth in four major do- 
mains: academic performance and expectations, motivations and beliefs, social behavior, and 
health. After a synopsis of the major findings, the chapter presents more detailed descriptions of 
the impacts on child outcomes and discusses how these outcomes may have come about. 

Overall, children in New Hope families showed substantially better academic perform- 
ance, somewhat higher levels of positive social behavior, and lower levels of negative behavior 
than did their control group counterparts at both the two- and five-year assessments, suggesting 
long-term effects on children’s developmental pathways. The positive effects were strong and 
consistent for boys but were weaker and more mixed for girls. There were no impacts on ado- 
lescents’ risky behavior. 



Key Findings 

• Children in the New Hope program group performed better than those in 
control group families on several measures of academic achievement. They 
scored higher on a nationally standardized test of reading achievement, and 
program group parents reported higher reading and literacy skills for their 
children. These patterns occurred for both boys and girls and across the age 
ranges studied. Teachers rated New Hope boys, but not New Hope girls, 
higher on achievement than control group children. 

• Parents in New Hope families rated their children higher on positive social 
behavior than did parents in control group families, but there were not sig- 
nificant impacts on problem behavior. There were no overall effects on 
teachers’ ratings of social behavior or on children’s reports of delinquent ac- 
tions or other risky behavior. 

• Impacts of New Hope were larger and more consistent for boys than for girls, 
especially when teacher reports were considered. Teachers rated New Hope 
boys higher than control group boys on academic performance, appropriate 
classroom behavior, and positive social behavior. New Hope boys responded 
to hypothetical scenarios of peer provocation with less hostility than did con- 
trol group boys. Teachers rated girls fi'om New Hope families more nega- 
tively than they did control group girls. They rated program group girls lower 



on appropriate classroom behavior and positive social behavior and higher on 
problem behaviors. 

• Boys in the program group reported higher educational expectations and 
more engagement with school than did control group boys; there were not 
positive impacts on these measures for girls. 

• Effects on children’s psychological well-being were limited to adolescents 
(ages 13 to 16). Specifically, adolescents in New Hope families expressed 
more feelings of efficacy, greater engagement in school, higher expectations to 
attend and complete college, and stronger beliefs about the importance of 
community in their future lives than did adolescents in control group families. 

• There were no program impacts on peer relations or on risky behavior for ei- 
ther gender. 

• There were few program impacts on health except in families who entered 
the study with no barriers to employment. Those in New Hope reported that 
their children had better health and had more recent routine medical visits, 
but the children also had more behavior problems. 

• For the most part, program impacts did not differ for children in different 
ethnic groups or for those whose families entered the study with different 
employment histories or barriers to employment. 



The Conceptual Model 

The conceptual model for predicting child outcomes is shown in Chapter 1 (Figure 1.1). 
The New Hope program was expected to increase parents’ employment along with increasing 
income and work supports in the form of health and child care assistance. These policy compo- 
nents were expected to increase the family’s material resources, produce changes in parents’ 
time and social resources, and lead to improvements in parents’ psychological well-being. 
These aspects of the New Hope program were expected to affect children’s experiences within 
and outside the family: the nature of the parenting and home environment, the type and quality 
of child care, and children’s time use and activities. These, in turn, were expected to improve 
children’s educational progress, psychological well-being, and social behavior. This model al- 
lowed New Hope to affect families through diverse pathways, depending on their needs and 
circumstances, as illustrated in Box 6.1. 
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Box 6.1 



New Hope Assisted Families Through Diverse Pathways 

The New Hope Ethnographic Study (NHES) suggests that there was no single, pre- 
dominant pathway through which the program assisted families and indirectly supported chil- 
dren’s development.* Rather, New Hope seemed to benefit different families in different 
ways, reflecting the wide diversity among the families who constitute the “working poor.” 
Most commonly. New Hope was helpful when it offered benefits that fit into the family’s al- 
ready-functioning daily routine. For example, child care vouchers helped parents who wanted 
to provide better-quality care (as they defined this) or who wanted to disentangle themselves 
from social networks that were not providing care well. Some parents found New Hope case 
representatives valuable allies in finding jobs and services. But many parents working swing 
shifts, nights, or two Jobs — or those who had good, stable home care — did not use child 
care benefits. Parents who had health benefits did not find that benefit useful (although some 
did, because it also helped them to pay for employer-sponsored insurance); some other par- 
ents wanted to use New Hope’s insurance but could not regularly provide the copayment. 
Other families simply found New Hope a help with the cascading problems that most work- 
ing-poor families faced. Income supplements, more stable Jobs, a community service Job, or 
some combination of New Hope and state benefits helped them and increased their family 
stability. The parents who had no employment barriers or only one were especially likely to 
find their own pathway to more regular, stable employment and better daily life for their fami- 
lies, which in turn benefited their children. 



*Weisner, Gibson, Lower, and Romich, 2002. 



Income Effects and Children’s Outcomes 

The findings at the two-year evaluation and the five-year findings reported in earlier 
chapters supported parts of the conceptual model. New Hope increased parental employment, par- 
ticularly in stable Jobs — an effect that was still evident at the five-year evaluation. New Hope 
also produced a modest increase in family income, although the advantage associated with being 
in the program group was diminished by the time of the five-year follow-up (two years after New 
Hope benefits ended). Nonetheless, some of the literature concerning income effects on children 
suggests that income at particular developmental periods may have lasting effects. 

Family Income. Some investigators have proposed that poverty in early childhood 
(ages 0 to 5) has particularly strong and lasting effects,' but data are scarce. Longitudinal re- 
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'Huston, 1991. 



search shows that family income in the early years (ages 0 to 5) is related to intellectual devel- 
opment and adult attainment more strongly than is income at later ages.^ There is also evidence 
that children in families who move out of poverty during the first few years of the child’s life 
show gains in performance on tests of cognitive abilities.^ Therefore, the increased resources 
produced by New Hope might have occurred at a developmentally important time for the 
younger children in the sample. 

Parents’ Employment. New Hope increased parents’ employment. Most of the re- 
search on low-income families suggests that young children’s cognitive and social development 
is more positive in families with employed mothers than in those with unemployed mothers.'' 
But much if not all of this difference is a function of preexisting differences between employed 
and unemployed mothers in terms of demographic attributes, skills, personality, and child- 
rearing practices rather than being a product of employment.^ Parents’ job quality is associated 
with positive outcomes for their families,® so the fact that New Hope increased stable long-term 
employment may have contributed to children’s development. 

At the other end of the age spectrum in this study, maternal employment may have 
some negative effects for adolescents.’ In recent experiments testing welfare-to-work policies, 
including New Hope, there were scattered negative effects on school performance and minor 
deviant behavior for children who were adolescents when their parents entered the programs, 
even when programs raised income and had positive effects on younger children.® 

Parenting. The conceptual model in Figure 1.1 shows changes in family income and 
parental employment affecting material resources, parents’ psychological well-being, and par- 
enting practices. New Hope families were slightly better off financially, and these resources 
could have contributed to stimulation and opportunities for children. There is abundant nonex- 
perimental evidence that income effects on children’s psychological well-being occur in part 
through their effects on parents’ well-being and parenting practices. New Hope had some mod- 
est effects on parents’ psychological well-being, including reduced depression at the five-year 
follow-up. New Hope parents also reported better control over their children’s behavior, and 
their relationships with sons, in particular, were more positive than were those of control group 



’Duncan, Brooks-Gunn, Yeung, and Smith, 1998; Duncan and Brooks-Gunn, 1997. 

’Dealing, McCartney, and Taylor, 2001; NICHD, 2002. 

‘‘Vandell and Ramanan, 1992; Zaslow, McGroder, Cave, and Mariner, 1999; Zaslow, Rabinovich, and 
Suwalsky,1991; Zaslow and Emig, 1997; Woods, 1972. 

’Zaslow, McGroder, Cave, and Mariner, 1999. 

^Parcel and Menaghan, 1997. 

’See Huston, 2002. 

®Bos and Michalopoulos, 2001; Gennetian et al, 2002a; Zaslow et al, 2002. 



ERIC 



- 141 - 



parents. Therefore, the program might be expected to have more positive effects on children’s 
well-being. 

Child Care. The New Hope child care benefit allowed program group parents to 
choose relatively expensive care, and in the first two years, they used formal child care and af- 
ter-school programs for their preschool and early-school-age children more than control group 
parents did. Even after the program subsidies ended. New Hope parents continued to place their 
children in child care centers and other organized programs, whereas control group parents 
more often used home-based care and care by siblings. In general, formal, center-based child 
care provides more educational opportunities than home-based care provides, and several stud- 
ies indicate that it leads to more advanced cognitive and language development than informal 
child care.’ However, in the New Chance study, the amount of time that children had spent in 
child care centers during the very early years of life accounted for some of the negative impacts 
on school readiness.'® Any benefits or costs associated with child care would be expected to oc- 
cur for younger children in the sample. By about fifth grade, most children no longer participate 
in organized child care. 

Out-of-School Activities. One reason for some of the negative effects of maternal em- 
ployment on adolescents could be that children who are “too old” for child care have less super- 
vision and structure when their parents are employed. Involvement in structured activities in 
school and the community is one means of providing supervision and opportunities for youth 
development. At both the two-year and five-year follow-ups, children fi"om New Hope families 
participated more fi^equently in these activities and were less likely to be in unsupervised set- 
tings, which could counteract some of the negative effects of not having a parent available at 
home during some of the out-of-school hours (including summer). A second reason may be that 
adolescents may be expected to care for younger siblings or to assume more household chores 
while their parents work — although there was no evidence at either the two-year or the five- 
year evaluation that this occurred. In fact, the 9- to 12-year-olds in New Hope families were 
significantly less likely to care for siblings than were their control group counterparts. 

Gender Differences 

Although different effects of the program were not predicted for boys and girls, at the 
two-year evaluation, the impacts on boys’ school achievement, educational goals, and social 
behavior were more pronounced and more consistently positive than were the impacts for girls. 
Although teachers reported higher levels of academic achievement and higher levels of positive 
social behaviors (social competence, compliance, and autonomy) for New Hope children than 



’Lamb, 1997; NICHD Early Child Care Research Network, 2000, 2002. 
'®Quint, Bos, and Polit, 1997. 
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for control group children overall, program impacts were larger for boys than for girls. Accord- 
ing to teachers’ reports, boys in program group families had higher achievement, better class- 
room behavior skills (working independently, following classroom rules, making transitions), 
more positive behavior, and fewer behavior and discipline problems than boys in control group 
families. There were few program effects for girls, and, in fact, teachers rated New Hope girls 
higher on externalizing behavior problems and disciplinary problems in the classroom. 

Parents in program group families also reported higher levels of positive social behavior 
for their sons. These effects were large and reliable. Boys in New Hope families had higher as- 
pirations and expectations for their future occupations and for advanced education than boys in 
control group families. 

The greater impacts on boys were interpreted in light of boys’ greater risk of academic 
and behavior problems in the elementary years. Within the control group, boys’ academic per- 
formance and social behavior were considerably lower and less positive than those of girls. The 
program impacts brought New Hope boys’ scores up to the levels already attained by girls in 
both the program and the control groups. 

There was some evidence that the increased resources available to families benefited 
boys more than girls. Ethnographic interviews indicated that parents were concerned about 
boys’ vulnerability to gangs and antisocial behavior, and so parents used the additional re- 
sources provided by New Hope to purchase goods and opportunities for their boys. Program 
group boys were more likely to be enrolled in extended daycare and in structured out-of-school 
activities that provided supervision and learning experiences. 

Developmental Domains 

Each of four major domains examined in this chapter — achievement, motivation and 
attitudes, social behavior, and health — is central to successful development. Children’s path- 
ways in these domains during early and middle childhood are important predictors of adolescent 
and adult success. School achievement in the early and middle grades is a strong predictor of 
ultimate school success, educational attainment, and emplo3mient in adolescence and adult- 
hood." Motivation and beliefs about the future affect school achievement. Positive social be- 
havior and positive peer relations are indicators of mental health in childhood, and both are im- 
portant predictors of social competence in adolescence and adulthood. By contrast, behavior 
problems — particularly externalizing problems, aggression, and delinquent behavior in middle 
childhood and adolescence — predict delinquency and aggressive disorders in adolescence and 



"Mussen, Conger, Kagan, and Huston, 1990. 
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adulthood.'^ Finally, physical health forms a basis for later health and for behavioral develop- 
ment. Children with difficulties in any of these domains may need parents’ time and attention, 
making it more difficult for parents to remain employed. New Hope helped some of these par- 
ents (Box 6.2). 



Data Sources 

The sample for the child outcome assessments is described in Chapter 2 (and Figure 
2.1). In-person surveys with parents and children were conducted in the family’s home. The 
parents provided information about their children’s achievement and social behavior, and chil- 
dren were given several standardized tests and questionnaires. Two age-appropriate versions of 
the children’s instruments were used — one for ages 6 to 8 and one for ages 9 to 16. Some in- 
struments were administered only to children age 9 and older, and a few questions were added 
for respondents who were 12 and older. These are denoted by an age in parentheses — for ex- 
ample, “(age 9+),” “(age 12+).” Some measures are identical to those used at the two-year 
evaluation and are noted as “(2&5).” Information on standardization samples, reliability, valid- 
ity, and intercorrelations among these instruments is provided in Appendix D. 

Teacher reports about children’s academic performance, classroom skills, school pro- 
gress, and social behavior were obtained by questionnaires mailed to the children’s school. 
Teachers were told that children and their families were participating in a study but not that 
families were involved in an evaluation of New Hope, welfare, or poverty-related programs. 

All the analyses compared the entire group of children in the survey sample of New 
Hope families with children in control group families. For each outcome, differences in impacts 
were examined for boys and for girls as well as for three age groups: 6- to 8-year-olds, 9- to 12- 
year-olds, and 13- to 16-year-olds. Because some of the economic impacts differed for families 
with different levels of employment at baseline, with different barriers to employment at base- 
line, and in different ethnic groups, child impacts were examined for these subgroups as well. 
There were almost no systematic differences for children in families with and without full-time 
employment at baseline, so those results are not presented. 



'^Huesmann, Lagerspetz, and Eron, 1984. 
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Box 6.2 

New Hope Eased Daily Life for Children with Chronic Troubles 



In the New Hope Ethnographic Study (NHES), nearly half the families had children 
who had significant and chronic behavioral, cognitive, or developmental “troubles” (for ex- 
ample, placement in special education, behavior problems at home or in school, poor school 
achievement, or chronic health problems).* How did parents in these families work, care for 
their children, and make ends meet, and how might New Hope have helped these families 
make more successful adaptations? Poor families generally are more likely than are higher- 
income families to have a child with these kinds of troubles, and poor parents are also more 
likely themselves to be living with disabilities of various kinds.^ Children’s problems make it 
more difficult for parents to make ends meet, to provide help for their children, and to retain 
employment. 

Katrina, who was in the control group, has four children, including 8-year-old Neil, 
who has significant behavioral problems. At one point, Katrina’s car broke down, but she was 
unable to have it repaired for two weeks because she had to wait for her next paycheck. Dur- 
ing those two weeks, she stopped going anywhere other than work, and the family lived on 
fi-ozen pizzas. In this same period, Katrina had to move Neil out of child care because he was 
biting people, and she had to find transportation to get her other three children to child care. 
She talked with the fieldworker about her finstrations concerning Neil: “I’ve got the other 
[three] kids, the world cannot just be him. . . . We gotta eat. I gotta work. I am really angry 
with him because he takes away fi*om everything. . . . He’s making it hell for everybody.” She 
said that nothing was easy with Neil; even eating dinner was an event. She wondered about 
increasing his daily medication but was not sure that it was a good idea: “I don’t want the boy 
to ‘veg’ out. I don’t want him to be a vegetable. He’s got to have some life.” 

New Hope had some positive impacts on family adaptation for those families who 
had children with chronic troubles. For example, Wendy’s son Jordan was in a special educa- 
tion program for children with emotional disturbances and had behavior problems at home as 
well. New Hope helped Wendy obtain a child care license and open up her own daycare pro- 
gram. “I’m hoping and praying to be stable with this occupation because of the kids,” she 
commented. She looked forward to being able to go down to the school more often when 
there was a problem with Jordan. 

Edith had a son. Max, who had a hearing problem and behavior problems at home and 
school. Thanks to the medical insurance offered by New Hope, she was able to have Max’s 
hearing problem treated surgically, and she found psychological counseling for him as well. 

A well-run antipoverty program can make a difference in the daily lives of families 
with children who have developmental and other problems. New Hope families and control 
group families were equally likely to have a child with troubles, but families in New Hope 
had fewer problems sustaining their work and managing their family routines. The improve- 
ment could not always be tied to specific benefits (health insurance, child care, or income 
supplements) but seemed to result fi’om the combination of New Hope’s benefits and services. 



♦Bemheimer, Weisner, and Lowe. Forthcoming, 2003. 

^Duncan, Brooks-Gunn, and Klebanov, 1994; Moore and Driscoll, 1997; Olson and Pavetti, 1996. 
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Children’s Academic Performance 



Measures of Academic Performance 

All measures of children’s academic achievement were administered to the entire age 
range studied. 

Standardized Achievement Test Scores. To assess reading and mathematical compe- 
tencies, children completed four individually administered scales from the Woodcock-Johnson 
Achievement Battery.'^ Two of these (Letter-Word Identification and Passage Comprehension) 
measure reading skills; the average of these two is the Broad Reading score. The other two scales 
(Applied Problems and Calculation) measure mathematics skills; the average of these two is the 
Broad Math score. The total score is the average of all four scales. The Woodcock-Johnson was 
selected because its normative sample is large and representative and because it includes children 
from diverse ethnic groups and diverse types of schooling. The standard score for each scale is 
obtained by comparing the child’s score with norms for his or her chronological age group. The 
mean standard score for the population as a whole is 100; the standard deviation is 1 5. 

Parent Ratings of Achievement. Parents rated their children’s overall level of achieve- 
ment on a five-point scale ranging from “below average” to “excellent” (2&5). Based on their 
knowledge of recent report cards, parents evaluated their children’s performance in reading, 
mathematics, and written work using five-point scales. 

Additional indicators of children’s school performance included parent reports of reten- 
tion in grade, remedial educational services, and participation in gifted programs. Parents re- 
sponded “yes” or “no” to a set of three questions about positive school experiences (whether the 
child had been in a gifted program or received school awards for academic or other types of 
achievement) and a set of questions about negative school experiences (whether the child had 
been in special education, repeated a grade, or received poor grades). For adolescents (12+), 
parents reported whether the child had dropped out of school before graduating. 

Teacher Ratings of Achievement. The teacher survey included the academic subscale 
of the Social Skills Rating System (SSRS), which was used in the two-year assessment (2&5).'‘‘ 
On this 10-item measure, teachers rated children’s performance in comparison to others in the 
same classroom on reading skill, math skill, intellectual functioning, motivation, oral communi- 
cation, classroom behavior, and parental encouragement. 



’^Woodcock and Johnson, 1990. 
'“'Gresham and Elliot, 1990. 



A “mock report card” completed by teachers indicated children’s current school perform- 
ance on reading, oral language, written language, math, social studies, and science. This measure 
was adapted from one used in the NICHD Study of Early Child Care and Youth Development.'^ 
Teachers also completed the Classroom Behavior Scale (2&5), which contains items concerning 
children’s study skills, conformity to classroom mles and routines, ability to work and complete 
tasks independently, and ability to make transitions without becoming distracted.'® 

Three items measured teachers’ educational expectations for the focal child. On a five- 
point scale ranging from “not at all” to “very ,” teachers indicated how sure they were that the 
child would finish high school, go to college, and finish college. These responses were averaged 
to produce one score representing the teacher’s expectations for the child’s educational attainment. 

Program Impacts 

Impacts for the Entire Survey Sample. New Hope had positive impacts on several 
indicators of children’s achievement. The children in the New Hope program group performed 
better on the broad reading score and on the total score for the Woodcock- Johnson test of 
achievement than did children in the control group, although the latter impact just misses statis- 
tical significance (p = .108). The means appear in Table 6.1. The effect size indicates that, on 
average, program group children scored .12 standard deviation higher than control group chil- 
dren on the total score and on the reading score. There were not significant group differences on 
math skills. 

New Hope program parents also reported higher reading and literacy skills for their 
children than did control group parents (Table 6.1). There were no overall impacts of the pro- 
gram on parent or teacher reports of positive indicators of school progress (such as gifted ser- 
vices and academic or other awards) or negative indicators of school progress (grade retention, 
remedial services, poor grades, or dropping out). There were no overall differences in teachers’ 
ratings of academic skills and behavior of program group and control group children, largely 
because teacher-rated impacts differed markedly for boys and girls. 

Impacts by Child’s Gender. Overall, the impacts of New Hope were more positive for 
boys than for girls (Table 6.2). Gender differences were most pronounced for impacts on 
teacher reports of academic skills and classroom behavior. Teachers rated program group boys 
significantly higher than control group boys on the SSRS academic subscale and on the Class- 
room Behavior Scale. The means for program group boys were .30 standard deviation higher 
than those for control group boys. At the same time, teachers rated program group girls lower 



'®This measure can be found at http://secc.rti.org. 
'®Wrightand Huston, 1995. 
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The New Hope Project 
Table 6.1 



Impacts on Children's Achievement and School Progress for the Survey Sample 







Program 


Control 






Effect 


Outcome 


Range 


Group 


Group 


Difference 


P-Value 


Size^ 


Woodcock-Johnson test 

of achievement^ 

Total standard score 




96.0 


94.2 


1.7 


0.108 


0.12 


Broad reading score 




98.1 


96.0 


2.0 * 


0.091 


0.12 


Letter-word score 




97.6 


94.8 


2.9 * 


0.056 


0.15 


Comprehension 




98.6 


97.4 


1.1 


0.338 


0.07 


Broad math score 




93.8 


92.5 


1.3 


0.250 


0.08 


Calculation 




90.2 


88.4 


1.7 


0.168 


0.09 


Applied problems 




97.9 


96.9 


1.0 


0.456 


0.06 


Parent ratings of 


l=below average, 












achievement 
Overall achievement 


5=excellent 


3.7 


3.7 


0.0 


0.854 


0.01 


Reading 




3.7 


3.5 


0.2 *** 


0.006 


0.19 


Math 




3.7 


3.6 


0.1 


0.455 


0.05 


Teacher ratings of achievement 
Social Skills Rating System (SSRS) 
academic subscale 


1 =lowest 1 0 percent 
of the class, 
5=highest 10 percent 
of the class 


3.2 


3.2 


0.1 


0.517 


0.06 


Mock report card - total 


l=below average, 
5=excellent 


2.9 


2.9 


0.0 


0.637 


0.04 


Mock report card - reading 


l=below average, 
5=excellent 


2.9 


2.8 


0.1 


0.387 


0.08 


Mock report card - math 


l=below average, 
5=excellent 


2.8 


2.9 


0.0 


0.814 


-0.02 


Classroom Behavior Scale 


l=almost never, 
5=almost always 


3.7 


3.7 


0.0 


0.821 


0.02 


Teacher expectations for child 


l=not at all, 
5=very 


2.8 


2.8 


0.0 


0.907 


0.01 


School progress 

Positive school progress: parent report 


0=no, l=yes 


0.4 


0.4 


0.0 


0.153 


0.11 


Negative school progress: parent report 


0=no, l=yes 


0.2 


0.2 


0.0 


0.391 


-0.06 


Time absent: teacher report 


l=less than 2 percent, 
4=more than 1 0 percent 


1.9 


1.8 


0.1 


0.265 


0.10 


Time tardy: teacher report 


1 =less than 2 percent, 
5=more than 25 percent 


1.6 


1.6 


0.0 


0.807 


0.02 



(continued) 
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Table 6.1 (continued) 



SOURCE: MDRC calculations using data from the New Hope five-year survey, 

NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Test results were available for 816 children; parent reports were available for 830 children; and teacher reports were 
available for 530 children. Actual sample sizes for individual measures may vary as a result of missing data, 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

^Woodcock-Johnson scores are age-standardized vvith a mean of 100 and a standard deviation of 15, 



than control group girls on both of these measures, although program-control group differences 
were significant only for classroom behavior. The means for program group girls were .17 stan- 
dard deviation and .27 standard deviation lower than those for control group girls on academic 
skills and classroom behavior, respectively. 

Program impacts on Woodcock- Johnson achievement scores and parent ratings were 
positive for both boys and girls. The size of the effects, however, were slightly larger for boys 
than for girls; as a result, in separate analyses, some program-control differences were signifi- 
cant only for boys. Program group girls showed the same pattern of better achievement scores 
than their control group counterparts, but the differences were not significant. 

Impacts by Child’s Age, For the most part, the positive impacts on children’s aca- 
demic outcomes occurred throughout the age range studied. For the oldest group of children 
(who were 8 to 10 years old at random assignment and 13 to 16 years old at the five-year fol- 
low-up), New Hope parents reported fewer negative school experiences than did control group 
parents. That is, the adolescents in New Hope families were less likely to be retained in grade, 
to be receiving remedial services, or to be receiving poor grades. Appendix Table E.6.1 shows 
New Hope’s impacts on children’s academic achievement and school progress across the three 
age groups. 

Impacts by Parents’ Barriers to Employment, Chapter 3 noted that the strongest im- 
pacts of New Hope on employment and earnings occurred for parents who had one barrier to 
employment when they entered the program (compared with those who had zero or many barri- 
ers). Therefore, the impacts on children in these three subgroups were examined to determine 
whether the positive impacts were concentrated in families with one barrier. For the most part. 



Impacts on Children's Achievement and School Progress, by Gender 



=3 w > 
^ C 

C <L> 
< 5 CQ 
CU W 



a 

T3 

C 

cd 

5/3 

>3 

O 

CQ 



UQ 



c 

o 

U 

E 

eo 

00 

o 



o 

c 






« 












♦ 




♦ 








o 




OV 


p 




ro 




00 


o 


CN 


O 


p 


— 


— 


— 


d 


— 


CN 


Tf 


— 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


5 






























o. 




OV 


CN 


OV 


p 


o 


m 


p 


m 


p 


p 


oo 


p 


p 


33 

Q 






rn 


vb 


rn 


d 




rn 


fb 


rn 


ri 


fN 


fN 


fN 




ov 


ov 


ov 


ov 


ov 


ov 


ov 
















a 






























a 


00 




(N 


p 


p 


ov 




p 


p 


p 


(N 


OO 


p 


p 


33 

Q 


VO 






00 


rn 


ov 


00 


rn 


rn 


rn 


rn 


fN 


(N 


fN 




OV 


ov 


ov 


ov 


ov 


00 


ov 

















a 



oou->ovor^Tf 
VO m O O VO Tf 
00 VO vn ov m CN 00 

d d d d d d o 



— 00 O 00 CN — Tf 

— — (N O O O O 

d d d d d d d 



VO — 00 Tf Tf 

00 o VO 00 

(N — O rr OO ov VO 

d d d d d d d 



VO Tf 

CN ov oo 
00 VO 



— o — 

O (N O 



VO 00 
m Tf 00 
Ov O 00 



o 

o 

d 



O 



OV 

Tf 



CN 

m 

VO 

d 



o — 

d d 



o 

d 



VO — — 

VO m VO 

d d d 



ra 

1^ 

u 

O W 

o 

O X 

.O U 

II II 

— VO 



o. 

o 



o. 

o 



> 'P > 'P > 'P 

t/3^_.w5Cd“COjrrrtJrr 



b 

> 

II Vvx II Vm II II II II II II II 

— O VO o — VO— VO — vovo 



_ o <D o ^ O ^ 
^.2J^a<L>x<L»X<L»X 



R Ol 

Vm g 

o g 

tS S 

" 2 a 2 

o 8 S 2 2 E “ 

|-SS.8§8 Jo 

o^*o;’Ocj=cO w) 

.S 

•g22»-gEi-a ^ 

o Jf T3 S.T3 3 .Si I- 

■oSgSEgiig. ^ 

OOk^oOucd — 

oHCQnJUCQU< 1= 

^ £ 



u 

E 

u 

> 

o 

IE 

u 

— Ofl 

2 ^ j= 

^ S « 

O Qi S 



T3 

= § 
E E 

o» U 

.2 >> 
,C c/3 
« CiO 
« C 



M 

D£ 



eq 

u c/3 

^ C3 

J5 O 

a o 

S? C/3 



OO 

c 



i2 

o 



^ u. 



-E 

cq 

o 

t; 

o 

o. 

u 

u, 

o 

o 



-E 

cq 

o 

t; 

o 

o. 

o 

k« 

o 

o 



T3 

u. 

R 

O 

t; 

o 

o. 

u 

V-. 

o 

o 



s s s 



- 150 - 



ERIC 



186 



■a 

o 

9 

C 

c 

o 

VC 

z 

H 



.2 2 c 

^ g g 

c > 

cL Q 



ffl 



PU 



1 i 

o o 

E §■ 

2 I 

P o 



O 

o 



VO 

m 



m *— 



vn 

fN 

o 

o 



Tf 

m 

d 



cs 

d 



CN 



O) 

CN 



1- >> 

U C4 

= ” « 

(A (A ^ 

O O cd 

E E B 

(d cs c 

II II II 

— ir» — 



o 

> 

ed 

u 

E 

o 

o 



« u 



CN 

00 00 

OO CN 



r» m 

m Tf 

— CN 



o 
c 

II II 

o o 



o 

c 



a. 

X 



o 

Tf 

00 



Tf (N — 



tn — 

^ o 

m O) 

d d 



o o o 

d d d d 



m m 00 vq 

d d — ^ ^ 



Tt CN On vq 

d d — ^ ^ 



M CO 
U U 

iTir 





c 




c 


c 

u 

2 


u 

2 

o 

p. 


c 

U 

2 


u 

2 

u 

a. 


u 

a. 


o 


u 

a. 


CN 


CN 


c 


CN 


c 


c 


Cd 


c 


cd 


Cd 


-C 


cd 


-C 






M 






u 




u 


CA 

CA 


k- 

o 


fA 

(A 


k- 

o 


O 


E 


O 


£ 


T 


II 


T 


iT 




Tf 







1- t 

2 

O. a> 

u V. 
Im 

C 
U 

Ui 

cd 

D. 



C 

0> 

b« 

cd 

D. 



. . CO 
CO CO 
CO u 

u c 
00 
00 o 

sjl 

Sd ^ 

1- w > 
o-.> -9 

* w ed 
O ‘co 
O O ^ 
-g Cl- Z 

C/3 



t: 

o 

D. 

O 



B 

H 



o 

o- 



T3 



E 

H 



U 



Tf — — o Tf lAi 
— O 

d d d d d d d 



OrnCsr^ — O — 

Tf VO 00 o m o CN 

*— CN CN m ^ ^ Tj; 

d d d d d d d 



o 

B 

a 

> 

o> 



S cj o 



C Cd 
2 2 



g-s 

^ o 



*o 

o 

e 



■^ § S E g 5 



CiJ h- u O h- CC3 W 

H m kj o m o < 



Cn CN 
— Cn 00 
OO O Tt 



o 00 vq cn 00 CN 
CN — ^ CN — ^ CN CN — ^ 



CN On CN vq Tf cq »rj 

Tf vd vd ^ vd 

CS CS CS On On 00 CS 



m m CN OO — 

vd 00 00 On rn d 
GS On On On On On On 



O CN — 

d d d 



Cn rq rq 
m m m 



On On OO 
m m m 



I s 

a X 

^ u 



4 ^ 

E 

u 

4 ^ 



o 










£ 


c 


o 

o 


u 

k- 




CJ 

k- 


CA 

E 

,2 


u 

eo 


d> 

E 


CA 


o 


C 


o 




CJ 


00 


u 


o 


u 


o 


> 


_c 


CA 

T 3 


CA 

c 


CA 

JZ 


Z 
c P 


(A 

00 


.2 

IB 



t5 O E. 

E E 

D. ^ 3 

'' D. 
D. 



0 ) 

L- 

eo 

CL, 



CJ 

= 

2 ^ 

4J cd ■9 

> S rt 
O Oi S 



- 151 - 



ERIC 



187 



(continued) 



Table 6.2 (continued) 
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Woodcock-Johnson scores are age-standardized with a mean of 100 and a standard deviation of 15. 



impacts did not differ across barrier groups, but the positive impacts on teacher-reported 
achievement were strongest in the families with one barrier. By contrast, the positive impacts on 
the Woodcock- Johnson achievement scores occurred in the families with more than one barrier 
to employment (Appendix Table E.6.2). 

Impacts by Ethnic Group. Impacts were similar for children from African-American, 
Hispanic, and white families (Appendix Table E.6.3). 



Children’s Motivation and Beliefs 

Measures of Motivation and Beliefs 

Competency Beliefs, Values, and Efficacy. Children were asked about their self- 
concept of ability, expectations for success, utility value, and attainment value for math and 
English/reading using items adapted from the Self and Task Perception Questionnaire.'^ Sample 
items include “How good at English are you?” and “How usefril is what you learn in math?” 
Responses were on a seven-point scale ranging from “not at all or a little” to “very.” 

Children’s sense of efficacy was measured using six items from the Children’s Hope 
Scale (age 9+), which has items that are similar to the adult Hope Scale completed by the par- 
ents.'® Each subscale includes three items using a six-point scale ranging from “none of the 
time” to “all of the time.” Sample items include: “I think I’m doing pretty well” and “Even 
when others want to quit, I know I can find ways to solve the problem.” 

School Engagement. Children’s perceptions of their school environment were assessed 
with five items (for example, “You feel close to others at your school,” “You feel like you are a 
part of your school”), using a six-point response scale ranging from “none of the time” to “all of 
the time” (age 9+). These items were adopted from the Adolescent Heath Survey. 

Aspirations, Expectations, and Values for the Future. Children ages 9 and older 
were asked to indicate how sure they were that they would finish high school, go to college, and 
finish college, using five-point scales (1 = “not at all sure,” 5 = “very sure”).'’ 

Children were asked about their occupational aspirations and expectations using a set of 
questions adapted from Cook and colleagues. They were first asked what job they would really 
like to have (aspiration), followed by what job they thought they actually would have (expecta- 



'^Eccles et al., 1983; Eccles and Wigfield, 1995. 
'®Snyderetal., 1996. 

’’Cooketal., 1996. 
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tion). Both responses were coded for prestige using updated scores developed by Nakeo and 
Treas based on occupational codes from the 1980 Census Occupational Classification.^® 

Children’s attitudes about work (for example, “You expect work to be a central part of 
your life”) were assessed with five items using a five-point scale from “strongly disagree” to 
“strongly agree,” taken from the Monitoring the Future Survey (age 9+). 

Children were also asked general questions about the importance of future achievements, 
using a five-point scale ranging from “not at all important” to “very important” (age 9+).^' These 
seven items asked about the importance of doing something to improve their community, being 
close to their family and raising kids, obtaining a good job, and having a lot of money. These 
items formed two scales — one referring to individual goals (for example, obtaining a good job) 
and the other referring to community involvement (such as improving their community). 

Program impacts 

Impacts for the Entire Survey Sample. There were no overall impacts on children’s 
motivation, attitudes about work, or feelings of efficacy (Table 6.3). 

Impacts by Child’s Gender. The impacts were different for boys and girls (Table 6.4). 
Boys in the program group had higher educational expectations and more engagement with 
school than did control group boys. The impacts for girls were statistically insignificant; how- 
ever, the pattern for girls contrasted with the pattern for boys. Program group girls expressed 
slightly less engagement in school and less expectation of graduating from college than did con- 
trol group girls. 

Impacts by Child’s Age. Most of the measures of motivation were given only to chil- 
dren age 9 or older. As shown in Appendix Table E.6.4, there were no significant program- 
control differences for 9- to 12-year-olds, but there were some positive impacts for adolescents 
(age 13 and older). Specifically, program group adolescents had higher educational expecta- 
tions, greater school engagement, stronger feelings of efficacy or hope, and higher expectations 
that they would contribute to their community in the future. 

Impacts by Parents’ Barriers to Employment and Ethnic Group. There were few 
differences in impacts on children when grouped by parents’ initial barriers to employment 
(Appendix Table E.6.5). There were few differences across ethnic groups, but there was some 
evidence that work attitudes and occupational expectations were reduced for white (non- 
Hispanic) children in New Hope families (Appendix Table E.6.6). 



^®Nakeo and Treas, 1994. 

^'These questions were adapted from Flanagan et al. (1998). 
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The New Hope Project 
Table 6.3 

Impacts on Children's Beliefs and Motivation for the Survey Sample 



Outcome 


Range 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size" 


Competency beliefs 
Child's self-perceived ability 














English 


l=not at all good, 
7=very good 


5.8 


5.8 


0.0 


0.683 


0.03 


Math 


l=not at all good, 
7=very good 


5.8 


5.8 


-0.1 


0.468 


-0.05 


Child efficacy (Hope Scale) 


l=none of the time, 
6=all of the the time 


4.7 


4.7 


0.1 


0.276 


0.10 


School engagement 

Aspirations and expectations 
Educational expectations 


l=none of the time, 
6=all of the the time 


4.0 


3.9 


0.1 


0.275 


0.10 


Complete high school 


l=not at all sure, 
5=very sure 


4.6 


4.6 


0.1 


0.394 


0.08 


Attend college 


l=not at all sure, 
5=very sure 


4.4 


4.3 


0.1 


0.172 


0.12 


Complete college 


l=not at all sure, 
5=very sure 


4.3 


4.1 


0.2 ♦♦ 


0.018 


0.22 


Occupational aspiration 


l=low, 100=high 


66.5 


66.0 


0.5 


0.748 


0.03 


Occupational expectation 


l=low, 100=high 


64.9 


64.0 


0.9 


0.585 


0.04 


Values for the future 


Work attitude 


l=strongly disagree, 
5=strongly agree 


4.3 


4.3 


0.0 


0.987 


0.00 


Future beliefs - individual 


l=not at all important, 
5=very important 


4.0 


3.9 


0.0 


0.414 


0.07 


Future beliefs - community 


l=not at all important, 
5=very important 


4.6 


4.5 


0.0 


0.442 


0.08 


Sample size 




413 


405 









SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 
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The New Hope Project 
Table 6.4 

Impacts on Children's Beliefs and Motivation, by Gender 
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BEST COPY AVAILABLE 



5=strongly agree 

Future beliefs - individual I =not at all important, 4.0 4.0 0.0 0.986 0.00 0.537 

5=very important 



Table 6.4 (continued) 
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Values for the future 

Work attitude l=strongly disagree, 4.2 4.2 0.0 0.701 0.05 

5=strongly agree 

Future beliefs - individual l=not at all important, 4.0 3.9 0.1 0.347 0.11 

5=very important 
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Children’s Social Behavior 



Measures of Social Behavior 

Positive Social Behavior. Most studies of children from low-income families empha- 
size the negative aspects of social behavior. This study gives equal emphasis to positive and 
problem behavior. Both parents and teachers completed the Positive Behavior Scale.^^ The 25 
items in it are divided into three subscales: compliance and self-control (for example, “Thinks 
before he/she acts,” “Usually does what I tell him/her”); social competence and sensitivity 
(“Gets along well with other children,” “Shows concern for other people’s feelings”); and 
autonomy (“Tries to do things for him/herself,” “Is self-reliant”). Both parents and teachers 
completed these scales. 

Problem Behavior. Both parents and teachers rated children on externalizing and in- 
ternalizing problems, using the Problem Behavior Scale of the Social Skills Rating System.^^ 
Externalizing problems include aggression and lack of behavior control (for example, “Is ag- 
gressive toward people or objects,” “Has temper tantrums”). Internalizing problems include so- 
cial withdrawal and excessive fearfulness (“Appears lonely,” “Acts sad or depressed”). Teach- 
ers also reported how often they had to discipline children for misbehavior. 

Peer Relationships. The Loneliness and Social Dissatisfaction Questionnaire is a 16-item 
scale measuring the child’s satisfaction with peer relations and fiiendships.^'* For 6- to 8-year-olds, 
the items are questions; and for 9- to 16-year-olds, they are statements (for example, “It’s hard for 
me to make new fiiends”). The child answers on a five-point scale, ranging from 1 (“always frue”) 
to 5 (“not true at all”). For this study, high scores indicate satisfaction with fiiendships. 

Hostile Intent Attribution. This measure is designed to measure children’s aggressive 
tendencies. It consists of four vignettes presenting situations in which another person does 
something that could be perceived as hostile (for example, “A kid spilled milk down your back 
while you were sitting in the school cafeteria”). Respondents are asked why the person did this. 
The choice of answers includes benign intent (such as “The kid slipped on something”) or hos- 
tile intent ( “The kid wanted to make fun of you”). Two of the stories involve physical hostile 
intent, and two involve social hostility (for example, not inviting someone to a party). Other 
research indicates that responses to this instrument predict children’s own aggressive behavior. 



^^Quint, Bos, and Polit, 1997. 

^^Gresham and Elliott, 1990. 

Asher and Wheeler, 1985; Cassidy and Asher, 1992. 
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Aggressive children are more likely to attribute hostile intent to others; nonaggressive children 
are more likely to attribute benign intent?^ 

Deviant Behavior and Peer Characteristics. A measure of Peer Group Conventional 
Behaviors, taken from the Pittsburgh Youth Study, was used to assess how many of the respon- 
dent’s close friends got good grades and participated in sports, school activities, and religious 
activities (age 9+). 

Trouble Index. Parents responded “yes” or “no” to five questions asking whether 
their child had been suspended from school, been in juvenile court, had a problem with alco- 
hol or drugs, gotten in trouble with police, or done something illegal to get money. For chil- 
dren older than 12, parents were also asked whether the child had gotten pregnant or gotten 
someone pregnant. 

Delinquent Behavior. This 15-item scale was used to assess self-reported deviant be- 
havior (age 9+).^® Youth were asked how often in the past 12 months they had engaged in fight- 
ing, stealing, vandalism, and substance use. The score is the total number of delinquent actions 
reported. For respondents age 12 and older, two questions were added, asking whether they had 
ever “had sex” and whether they had ever been pregnant or gotten a girl pregnant. 

Peer Group Deviant Activities. The foregoing items about delinquent behavior were 
also asked about respondents’ close fiiends. 

Program Impacts 

Impacts for the Entire Survey Sample. Table 6.5 shows that there were only 2 sig- 
nificant impacts out of 19 comparisons — findings that could have occurred by chance. Parents 
in New Hope families rated their children higher on positive social behavior than did parents in 
control group families, and New Hope children scored lower on hostile intent attribution than 
control group children. 

Impacts by Child’s Gender. There were some significant differences in impacts for 
boys and for girls (Table 6.6). Teachers rated New Hope boys more favorably on social behav- 
ior than they did control group boys, but the pattern was reversed for girls. The teachers’ posi- 
tive behavior scores for New Hope boys were .24 standard deviation higher than scores for con- 
trol boys; New Hope girls’ scores were .26 standard deviation lower than control girls’ scores. 



Crick and Dodge, 1996. 

^®The items were adapted from LeBlanc and Tremblay (1988). 
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The New Hope Project 
Table 6.5 



Impacts on Children's Social and Risky Behavior for the Survey Sample 



Outcome 


Range 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size“ 


Positive Behavior Scale 


l=never, 

5=all of the time 












Parent report 




3.9 


3.8 


0.1 * 


0.061 


0.15 


Teacher report 
Problem Behavior Scale 


l=never, 

5=all of the time 


3.6 


3.6 


0.0 


0.915 


0.01 


Total: parent report 




2.3 


2.4 


-0.1 


0.184 


-0.11 


Total: teacher report 




2.3 


2.3 


0.0 


0.650 


0.04 


Externalizing: parent report 




2.3 


2.4 


-0.1 


0.108 


-0.13 


Externalizing: teacher report 




2.1 


2.1 


0.0 


0.611 


0.05 


Internalizing: parent report 




2.4 


2.4 


0.0 


0.346 


-0.08 


Internalizing: teacher report 




2.3 


2.2 


0.0 


0.603 


0.05 


Disciplinary action: teacher report 




2.5 


2.4 


0.1 


0.511 


0.06 


Social relationships 


Peer relationships: child report 


l=always true, 
5=not true at all 


4.2 


4.1 


0.0 


0.739 


0.02 


Hostile intent total: child report 


0=benign, 

4=hostile 


3.0 


3.2 


-0.2 


0.113 


-0.12 


Hostile intent - physical 


0=benign, 

2=hostile 


1.1 


1.3 


-0.2 * 


0.065 


-0.13 


Hostile intent - social 


0=benign, 

2=hostile 


1.9 


2.0 


-0.1 


0.459 


-0.05 


Peer conventional behaviors 


l=none of them, 
5=all of them 


3.3 


3.3 


0.0 


0.831 


-0.02 


Risky behavior 


Trouble index: parent report 


0=no, l=yes 


0.1 


0.1 


0.0 


0.765 


-0.02 


Delinquent behavior: child report 


l=never, 

5=five or more times 


1.2 


1.1 


0.0 


0.263 


0.11 


Had sex: child report 


l=never, 

5=five or more times 


1.3 


1.4 


-0.1 


0.515 


-0.08 


Got pregnant: child report 


l=never, 

5=five or more times 


1.1 


1.0 


0.0 


0.653 


0.06 


Peer delinquency: child report 


l=none of them, 
5=all of them 


1.4 


1.4 


0.0 


0.837 


0.02 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Teacher reports were available for 530 children; parent reports were available for 830 children; and child reports 
were available for 840 children. Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even if the table shows impacts for subgroups. 
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Impacts on Children's Social and Risky Behavior, by Gender 



^ ^ ^ 

« ^ I 

i'Q® 



a 

c 

cd 

c/3 

>1 

O 



£ 



c 

o 

U 



o. 

p 

e 

a 



p O 



— »n 
00 o 
00 o 
o o 



m Tf 
fS 



r- 00 
o r- 
fS o 

C) C) 



fS 

o o 



> ° 
U PZ 
C ed 
I! II 

« 1^3 



a 

o 



o 

Cu 



O 00 VO CN ^ O O 
r«- fs ^ ^ rn 00 00 

o VO cN o ^ 
o o o o o o o 



m CN m Tf o VO 
— — —0 — 00 
o o o o o o o 



VO Tf o VO OV O VO 

m fs CN m VO m 

(N m (N rn VO 

o o o o o o o 



o o o o o o o 



Tj; Tj; -rj; — Tt cn 
H fS CN CN fS fS fS 



rn rn rn — fN VO 
^^i r4 fs CN cs fs fs 



u o 

U PZ 

c cd 



t: 
t: o 
o Cl, 
a, H 

“ D 



Z cd 
cd w 
Oh H 



.2 

*> 

A 

J= 

0> 

P9 



^ 

^ 2 - 



5 S *-= 



S E E 



o 



o 

00 

m 

O 



OV 

o 

o 



VO 

OV 



OV 

o 



CN 

o 



o 

c> 



— 



— 



OV 

o 



Tf 

CN 



o 

o> 



CN 

O) 



— CN 



fN 

Tf 



SaSaEaEa 

0) (L) ^ 1) ^ 1) 

c/] = c/] = cy 3 — C/J = 

^E ^E ^E ^E 

cd'^cd'^ 

ACACACAC 

II II II II II II II II 

— m — — m 



t: 

o 

Qu 

u 

Im 

“1 
. . 

-c c 

.2 

0) ^ 



cn 



c 

o 

Qu 



2 

2 

u 

lo 



t: 

o 

D. 






u- z; N N jn 

A g ;= ;= .N .N 
O. «„• ra A *3 »— — "7^ 



E 

S ^ 
2 5 



.2- ^ 
2 t/i 

e c 
.2 .2 
A _A 



>s 

JZ 

Cl 



O 

*> 

A 

-C 



is is u 

•S .s g 

g> o O 



.2 g 

S 

C/D 



O 



o 



o 

X 



r. 

o 

a 



k. X 
.2^ 
^.E 

JS U 

-1 

o 

^ u 

2 H 



- 163 - 



ERIC 



1^9 



l=noneofthem, 3.4 3.4 0.1 0.371 0.12 0.077 f 

5=all of them 
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The same pattern appears on teachers’ ratings of problem behaviors. New Hope boys’ 
ratings were .12 standard deviation lower than control boys; New Hope girls’ ratings were .27 
standard deviation higher than control girls’ scores. 

On the hostile attribution measure, New Hope boys scored .25 standard deviation lower 
than control boys; there was no program impact for the girls. 

There were no program impacts on peer relations, delinquency, or sexual involvement 
for either gender, with one exception: New Hope girls reported fewer conventional behaviors 
(for example, interest in good grades, participation in school activities) among their peers than 
did control group girls, but this impact just misses statistical significance. 

Impacts by Child’s Age. There were few variations in program impacts for different 
age groups (Appendix Table E.6.7), and those that did occur were all based on parent reports. 
New Hope parents reported slightly more internalizing problems for 6- to 8-year-olds than did 
control group parents; for older children, however, parents reported fewer problem behaviors — 
particularly fewer internalizing problems (9- to 12-year-olds) and more positive social behavior 
(13- to 16-year-olds). 

Impacts by Parents’ Barriers to Employment. Most of the impacts on children’s so- 
cial behavior did not differ according to parents’ barriers to employment, but the positive effects 
of New Hope on children’s perceptions of hostile intent were greatest in the families who had 
many barriers (Appendix Table E.6.8). 

Impacts by Ethnic Group. On average, impacts on social behavior were slightly more 
positive for African-American children and were more mixed for white children (Appendix Ta- 
ble E.6.9). 



Children’s Health 

Measures of Health 

General Medical Conditions. Parents were asked whether any children in their family 
had “health problems requiring frequent trips to the doctor”; “behavior that is hard to control”; 
and/or “any other health condition, problem, or disability.” One question asked parents to rate 
focal children’s overall health, using a scale from 1 (“poor”) to 5 (“excellent”). 

Health in the Past Year. Parents were asked whether, at any time in the past year, a 
focal child had “been a patient in a hospital overnight”; “had a health problem, disability, or 
behavioral condition that had been diagnosed by a medical professional”; “received any pre- 
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scription drug for a health or behavioral condition”; and/or “received any counseling or therapy 
for emotional, mental, or behavior problems.” 

Health Care Facilities. Parents were asked, for focal children, whether there were “a 
particular health clinic, health center, doctor’s office, or other place that you usually go to if 
[your child] is sick or you need advice about (his/her/their) health.” They were also asked: “Is 
there someone at the places you usually go to who knows about your (child/children’s) health 
history who will give you advice over the telephone?” 

Time Between Medical Visits. Parents were asked, for focal children: “About how 
long has it been since (your children/each of the children) last saw a medical doctor or other 
health professional for a checkup, shots, routine care, or because the child was sick?” A six- 
point scale was used for responses: 1 = never; 2 = three or more years; 3 = at least two years but 
less than three years; 4 = at least one year but less than less than two years; 5 = between six 
months and one year; and 6 = less than six months. Using the same scale, parents were asked: 
“About how long has it been since (your child/each of the children) last saw a dentist or a dental 
hygienist for dental care?” 

Program Impacts 

Impacts for the Entire Survey Sample and by Child’s Gender. There were no sig- 
nificant impacts on the health measures for the entire survey sample (Table 6.7), and no differ- 
ences were foimd between boys and girls (not shown). 

Impacts by Child’s Age. Among children ages 13 to 16, those in the program group 
had significantly better reported health than did those in the control group (Appendix Table 
E.6.10). 



Impacts by Parents’ Barriers to Employment. Impacts for the families with no barri- 
ers to employment were different from impacts for families with one or more barrier. Specifi- 
cally, for families with no barriers. New Hope families reported that their children had better 
health, had been less likely to spend the night in a hospital, and had had a shorter time period 
since seeing a doctor or a dentist for routine care than did control group children. In the families 
with no barriers to employment. New Hope parents also reported that their children had more 
behavior problems but less coimseling (Appendix Table E.6.1 1). 

Impacts by Ethnic Group. There were no consistent differences in impacts across eth- 
nic groups (Appendix Table E.6.12). 
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The New Hope Project 
Table 6.7 



Impacts on Health for the Survey Sample 



Outcome 


Range 


Program 

Group 


Control 

Group 


Difference 


P-Value 


Effect 

Size“ 


General medical conditions 
General health problems 


l=no, 2=yes 


1.3 


1.2 


0.0 


0.214 


0.11 


General behavior problems 


l=no, 2=yes 


1.2 


1.2 


0.0 


0.373 


0.08 


Other health condition 


l=no, 2=yes 


1.2 


1.1 


0.0 


0.722 


0.03 


Overall health 


1 =poor. 


4.3 


4.2 


0.1 


0.390 


0.07 


Health in past year 
Overnight hospital stay 


5=excellent 
l=no, 2=yes 


1.0 


1.1 


0.0 


0.633 


-0.03 


Diagnosed illness 




1.2 


1.2 


0.0 


0.385 


0.06 


Received prescription drug 




1.2 


1.2 


0.0 


0.389 


-0.07 


Received counseling or therapy 




1.1 


1.1 


0.0 


0.307 


0.08 


Health care facilities 

Used particular health care facility 


0=no, l=yes 


1.0 


1.0 


0.0 


0.473 


-0.06 


Received medical advice over phone 




0.9 


0.9 


0.0 


0.402 


0.08 


Time between medical visits 
Time since last doctor visit 


1 =never, 

6=less than six months 


5.3 


5.4 


-0.1 


0.149 


-0.12 


Time since last dentist visit 




4.8 


4.8 


0.0 


0.686 


-0.04 


Sample size 




429 


421 









SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 
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Comparison with Two-Year Results 

Many of the impacts observed at the two-year evaluation persisted after five years. The 
positive effects of the New Hope program for children’s academic ftmctioning and progress that 
were evident at the two-year follow-up persisted through five years after random assignment — 
and, perhaps more important, for approximately two years after the program ended. Children 
fi’om program group families — and boys in particular — were performing better academically 
than children fi’om control group families. These findings are strengthened by the fact that they 
appeared across multiple data sources, including standardized test scores as well as teacher and 
parent reports of achievement. Test scores and parent reports suggest that effects were more 
pronounced for reading skills than for math. 

At both evaluations, positive impacts were stronger for boys than for girls. To illustrate 
the patterns over time, the means for boys and for girls at the two time periods were compared 
for some of the measures that were identical at the two evaluations (Figures 6.1 and 6.2).” 
These comparisons make it clear that control group boys were considerably lower on academic 
performance and classroom behavior than program group boys or than either group of girls at 
both time periods and that there was little change in the gap between program group and control 
group boys between two and five years. At both two and five years, program group boys had 
higher expectations for their future educational attainment, but the higher occupational expecta- 
tions found at two years did not persist in the later evaluation. 

At two years, there were large positive impacts on boys’ social behavior — both posi- 
tive and problem behavior — as reported by teachers. Some of these impacts endured, but the 
magnitude of the differences were much smaller. Boys continued to show significant positive 
impacts of New Hope on positive social behavior, although the size of the effects was dimin- 
ished fi’om .50 at two years to .24 at five years. The impacts on behavior problems — which 
also had an effect size of .50 at two years — had faded to nonsignificant levels by the five-year 
follow-up, and there was no evidence of impacts on adolescent deviant behaviors. Figures 6.3 
and 6.4 illustrate the patterns over time, making it clear that much of the reduction in impacts 
was due to improvement by control group boys. 

The negative impacts on teachers’ perceptions of girls, which were suggested in the 
two-year evaluation, were more pronounced at the five-year follow-up. The patterns are illus- 
trated in Figures 6.1 through 6.4, which make it clear that girls in both groups generally scored 
higher than boys. Nonetheless, girls in the program group, relative to those in the control group. 



^^Means in Figures 6. 1 through 6.4 are slightly different from those shown in the tables because these il- 
lustrations include only those children who had scores at both time periods. 
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The New Hope Project 
Figure 6.1 

Teachers' Ratings of Academic Achievement, by Child's Gender 
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SOURCES: The New Hope two- and five-year surveys. 
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The New Hope Project 
Figure 6.2 

Teachers' Ratings of Classroom Skills, by Child's Gender 
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Rating 



The New Hope Project 
Figure 6.3 

Teachers’ Ratings of Positive Social Behavior, by Child’s Gender 
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Rating 



The New Hope Project 
Figure 6.4 

Teachers* Ratings of Problem Behavior, by Child's Gender 
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had less favorable scores on all the teacher-reported scales, and these negative impacts in- 
creased in magnitude from two years to five years. 



Conclusion 

Summary of Impacts 

On the whole, children in New Hope program group families continued to fare better 
than those in control group families five years after their parents entered the program and two 
years after the program ended. They were making better academic progress (according to stan- 
dardized achievement tests, parent reports, and teacher reports), and they displayed slightly 
more positive social behavior. Those who were in elementary school when their parents entered 
the program and had reached adolescence were more engaged with school, had higher hopes 
about their futures, had higher educational expectations, and expressed a greater desire to con- 
tribute to their communities in the future. 

As was the case at the two-year follow-up, the program impacts were stronger and more 
consistent for boys than for girls. Most other studies of the effects of welfare reforms on chil- 
dren have not reported gender differences in impacts — which raises questions about whether 
there was something atypical about the New Hope sample. This question has been addressed in 
several ways. First, the large gender differences in experimental impacts in New Hope appeared 
primarily on teacher reports; for the most part, there were not significant gender differences in 
impacts for parent reports and test performance. Few other studies have included teacher re- 
ports, but a comparison with those that did indicates that gender differences are not unique to 
New Hope. Almost identical findings appeared in the New Chance study. For children ages 5 to 
9 at the follow-up, there were positive program impacts on teacher ratings of positive behavior 
and problem behavior for boys but negative impacts for girls.^® 

Second, teachers generally have more favorable impressions of girls than of boys. In 
New Chance, as in New Hope, they rated control group girls much more favorably than control 
group boys. Parents’ ratings were less differentiated by gender in both studies (and in other 
similar studies). The most widely used scale of behavior problems — the Child Behavior 
Checklist — has published norms for teacher ratings of boys and girls. The average score on 
externalizing behavior problems is higher for boys than for girls.^’ 



^®Quint, Bos, and Polit, 1997. 
^’Achenbach, 1991. 



Could the gender differences be a result of initial differences between control group and 
program group children? The random assignment design makes it unlikely that there were initial 
differences, but this notion was tested by comparing program and control group boys on 33 
baseline variables. One would expect 10 percent of these comparisons to reach a significance 
level of .10; in fact, 4 out of 33 (12 percent) reached this level. Two out of 33 (6 percent) 
reached a level of .05; one would expect 5 percent to reach this level by chance. In a similar 
comparison for girls, 1 out of 33 (3 percent) reached a probability of less than .10. The most 
parsimonious interpretation of these results is that there were not initial systematic differences 
between boys in families assigned to the program and control groups. Moreover, all impact 
analyses included controls for baseline characteristics — a further protection against bias by 
initial characteristics. 

Social Significance and Validity of Impacts 

New Hope’s impacts on several important child outcomes are statistically significant, 
but are they also socially and economically significant? Because many of the measures in this 
section contain scales that have different reference points, all tables show effect sizes as one 
means of estimating the magnitude of the program impacts. Cohen provides one way of under- 
standing what an effect size means: .10 is considered small; .30 is medium; and .50 is large.^® 
An economic perspective compares the value of the impacts to the costs of producing them. 

Using Cohen’s criteria, many of the impacts of the New Hope program were “small” to 
“medium.” Given the long time period between the program and the measured outcomes, these 
differences are sufficiently large to be socially important. An effect size of .12 on the reading 
achievement test’s standard score, for example, indicates that the average child in New Hope 
scored above 54 percent of the children in the control group. The average boy in a New Hope 
family scored higher than 57 percent of the boys in control group families. 

The New Hope impacts are also impressive when compared with the impacts of inten- 
sive early interventions designed specifically to improve school performance of low-income 
children. The Abecedarian program, for example, provided full-day, high-quality child care 
from infancy until school entry; it was tested with a random assignment experimental design. At 
age 12, the children in the intervention group scored 5 to 7 points higher than control group 
children on the Woodcock-Johnson scales of reading and math achievement.^' The children in 
New Hope scored approximately 1.5 to 3.0 points higher than control group children. Hence, 
New Hope produced about one-third of the gain that occurred in a very intensive and expensive 
early intervention program. 



^“Cohen, 1988. 
^'Ramey et al., 2000. 
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The impacts on children’s achievement are particularly persuasive, because they ap- 
peared on measures obtained from multiple sources: standardized tests, parents, teachers, and 
the children themselves. Teachers were given no information about children’s participation in 
New Hope or other interventions, so the program-control group differences on the teacher rat- 
ings are unlikely to have been affected by knowledge of the intervention. Impacts also occurred 
for answers that children themselves provided. In short, the program impacts are real, and they 
are large enough to be socially significant. 
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Chapter 7 

Conclusions and Implications 



New Hope was an innovative program designed to encourage work, to reduce poverty, 
and to demonstrate effective policies for working-poor adults and families. Its underlying prin- 
ciples were that people who are willing to work full time should be able to do so and that they 
should not be poor when they do. The program — in existence from 1994 through 1998 — con- 
sisted of four key components: job assistance, including referral to a wage-paying community 
service job when necessary; an earnings supplement for full-time workers that was designed to 
raise their income above the poverty level; subsidized health insurance; and subsidized child 
care. High-quality staff services were available in an atmosphere of respect and encouragement; 
project representatives offered participants assistance in conducting job searches, finding child 
care, and solving other employment-related problems. An important feature of New Hope was 
that it let working parents choose among a suite of benefits, to be used according to family 
needs and preferences. 

The initial New Hope model did not envision time limits, that is, a time when the pro- 
gram’s support for work would end. Funding constraints dictated this change in the original 
model. But because the New Hope offer did, in fact, expire after three years while the analyses 
presented here represent data that were collected five years after random assignment, this report 
concerns the lasting impacts that are associated with a time-limited model. 

New Hope was not designed as a “welfare-to-work” program but, rather, as a program 
to demonstrate the effects of supports for work among low-income persons, regardless of family 
status. As such, the intervention was intended to increase adult employment and income; it was 
not focused directly on children. In fact, however, strong lasting effects of the program occurred 
for children. 

Five years after their parents entered the New Hope program, children performed better 
on academic achievement — particularly reading — as measured by three independent sources: 
standardized test scores, parent reports, and teacher reports. There were some positive impacts 
on children’s motivation and social behavior as well. A summary of New Hope’s impacts on 
children and families appears in Table 7.1. Although the absolute effects are not large, they are 
sufficiently large to be socially important, given the long time period between the program and 
the measured outcomes. The average child in New Hope scored above 54 percent of the chil- 
dren in the control group on a standardized reading test. The long-term gains in achievement 
that were produced by New Hope were about one-third the size of the gains produced by the 
Abecedarian program, a very intensive and expensive early intervention program that was im- 
plemented in the 1970s. 
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The New Hope Project 
Table 7.1 

Summary of New Hope’s Impacts 


Outcome 


Program Group Versus Control Group 


Parents’ employment and income 


Modestly higher income 
Less poverty 
More stable employment 
Higher wages 


Parents’ well-being 


No difference in material or financial well-being 

Slightly better physical health 

Fewer depressive symptoms 

Better awareness of public and community resources 

Better able to sustain daily routine 


Parenting 


Few overall effects 

Fewer problems with control in discipline situations 
Boys: More positive parent relations 
Adolescents: More effective child management 


Child care 


More center-based care 
More after-school programs 
Less home-based care 
Less unsupervised care 
Fewer changes in arrangements 


Children’s out-of-school activities 


More participation in religious activities and organizations 
Adolescents: More participation in structured activities (for example, 
sports, lessons, community centers) 


Children’s academic achievement 


Better scores on standardized reading achievement test 
Better reading performance (as reported by parents) 
Boys; Better academic skills (as reported by teachers) 



Children’s motivation and well-being No overall impacts 





Boys: Higher educational expectations 

Greater school engagement 
Adolescents: 

Higher educational expectations 
Greater school engagement 
Increased feelings of efficacy to reach goals 
Greater future community involvement 


Children’s social behavior 


More positive social behavior (as reported by parents) 

No difference in risky, delinquent behavior 

Boys: More positive social behavior (as reported by teachers) 

More appropriate classroom behavior (as reported by teachers) 
Less hostility in provocation situations 
Girls: Less positive social behavior (as reported by teachers) 

More problem behavior (as reported by teachers) 


Children’s health 


No impacts 
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How and why did New Hope lead to lasting gains for children? Attempts to answer this 
question examined both direct and indirect pathways. The program was designed to increase 
parents’ employment and families’ material resources, and it did. Although the impacts on em- 
ployment and income faded somewhat after Year 3, New Hope participants had more stable 
employment, lower rates of poverty, and higher wages at the five-year point. Stable employ- 
ment and modestly higher income may have contributed both to resources for children (for ex- 
ample, center-based child care) and to parents’ psychological well-being. 

Compared with the control group, program group parents reported better physical 
health and slightly lower levels of depressive symptoms — both indicators of adult well-being. 
Perhaps more important, there was some evidence that program group parents were more aware 
of community resources and of the Earned Income Tax Credit (EITC). Ethnographic data sug- 
gest that a significant number of families intentionally used the EITC as a savings plan for mak- 
ing major purchases, reducing debt, and stabilizing rent and other payments. 

Changes in parents’ employment and family income are likely to affect children 
through their impacts on everyday experiences at home and away fi-om home. The initial model 
for New Hope identified parenting and parent-child relations as well as environments outside 
the family (such as child care and structured activities) as the likely pathways. Although there is 
a great deal of evidence showing that income affects parents’ well-being, which, in turn, con- 
tributes to positive parenting, this study found very modest evidence for program impacts on 
parental warmth, disciplinary practices, or ability to provide effective management of children’s 
behavior. Program group parents felt more in control of their children and provided more effec- 
tive child management (for example, infi’equent discipline), especially for their adolescents. 
Program group boys and their parents had somewhat more positive relationships and better 
communication than boys in control group families. 

These modest positive changes in parenting could partly account for children’s better 
academic and social behavior, but parents’ experiences of control could, of course, also be a 
result of the children’s more positive social behavior and better school performance. Ethno- 
graphic evidence suggests that positive and trusting relationships with children mediated the 
ability of parents to monitor their children while they were working (for example, by phone). In 
this context, it is noteworthy that program group boys reported higher levels of parental moni- 
toring than did control group boys. Recent studies show that parental monitoring of children’s 
whereabouts, fiiends, and activities is at least as much an indicator of children’s willingness to 
share information and to communicate with their parents as it is an indicator of parents’ vigi- 
lance and supervisory efforts.' The more positive communication and relationships between 



'Kerr and Stattin, 2000. 
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parents and boys may have enhanced parents’ ability to maintain employment without com- 
promising their family responsibilities. 

Parenting involves more than direct interaction with children; parents affect their chil- 
dren by the arrangements they make for children’s experiences in school, the community, and 
other settings. This aspect of parenting is often referred to as “family management.” New Hope 
had strong impacts on children’s experiences outside the family over the entire five-year period. 
Even though New Hope child care subsidies ended after three years, parents continued to use 
more formal center-based and after-school child care during the school year and more formal 
care during the summer. By contrast, control group children were more likely to be unsuper- 
vised and to be cared for by a minor during the summer. In addition, ethnographic data demon- 
strate that New Hope families had more stable child care arrangements (that is, fewer unex- 
pected changes in child care) than did control group families in the final year of the program 
and for about a year thereafter. 

Stable center-based child care and after-school programs have been shown repeatedly to 
contribute to children’s academic performance and to the development of social skills. In a 
large-scale longitudinal study, children who attended child care centers performed better on 
academic and cognitive skills than did those who experienced home-based child care, even 
when the quality and amount of care were controlled.^ Similarly, low-income children who at- 
tend after-school programs perform better in school and have more positive social behavior than 
those who do not.^ Among low-income children in the New Chance study, stable child care 
predicted school readiness and positive social behavior.'* Stable center-based care can be a sup- 
port for, as well as a consequence of, stable parental employment. Conversely, experiences in 
home-based and unsupervised care may have less salutary results, such as watching more tele- 
vision or hanging out with unsupervised peers, and it is often less reliable, making it a less 
effective support for parental employment. 

It is striking that New Hope program parents continued to use formal child care after 
their eligibility for child care benefits had ended and that older children continued to participate 
in more structured activities. One reason may be that program group parents had gained more 
sophistication about the public and private resources available to them. This knowledge may be 
one enduring legacy of the high-quality information and assistance that parents had received 
from New Hope’s project representatives. They may have been more proactive than control 
group parents in using a range of programs and services, particularly for older children. For ex- 



^NICHD Early Child Care Research Network, 2000, 2002. 
^Posner andVandell, 1999. 

‘*Bos and Granger, 1999. 
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ample, program group children participated in more religious classes and activities, suggesting 
that families made use of the resources offered by churches and other religious institutions. 

New Hope parents had more positive attitudes about the value of organized activities 
for their adolescents, and their teenage children were more likely to participate in a range of 
such activities. These experiences may have contributed to children’s social skills and to their 
motivation and interest in school. Adolescents in New Hope families had higher levels of school 
engagement, feelings of efficacy about achieving their goals, expectations to attend and com- 
plete college, and an expectation to contribute to their communities. Moreover, it is noteworthy 
that the impacts of New Hope on positive social behavior were slightly more consistent and 
lasting than were the impacts on problem behavior, particularly for adolescents. One of the con- 
tinuing issues in developing programs for young people is the tension between positive youth 
development and prevention of negative behavior. This study’s findings suggest that the experi- 
ences generated by participation in New Hope contributed to positive youth development — 
which is defined as more than simply the reduction of problem behaviors. 

The sustained impacts of New Hope on children’s academic performance may have re- 
sulted from the lasting effects of the program on children’s environments at home and away 
from home. But the lasting effects may also have resulted from advantages accrued during the 
three-year benefit period, which led to an upward spiral. That is, the better school performance 
(as rated by teachers) that children demonstrated at the two-year evaluation could have led to 
experiences of success, positive attitudes about school, and positive perceptions by teachers that 
were self-perpetuating. The initial treatment-induced changes in children’s behavior may also 
have affected the home and school environments that they experienced, either by eliciting par- 
ticular reactions from the people around them or by leading the children to seek out different 
activities, settings, and peers.’ 

In both the two-year and the five-year evaluations, the program impacts were more con- 
sistently positive for boys than for girls. Ethnographic data described in the two-year report 
suggest that parents perceive more dangers facing their boys than their girls and that many fami- 
lies used the additional resources from New Hope to provide boys with positive experiences and 
environments. For example, during the two years after random assignment, program group boys 
were more likely than girls (and control group boys) to be in after-school child care and struc- 
tured activities. Although the positive impacts on parent-child relations continued to be slightly 
greater for boys than for girls, the impacts on child care and activities outside the home were 
generally similar for girls and boys; hence, these experiences do not appear to account for the 
relatively large gender differences in impacts on the children. 



’Entwisle, Alexander, and Olson, 1997; Scarr and McCartney, 1983. 
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The gender differences in impacts should be considered in light of boys’ greater vulner- 
ability and higher risk of school failure. In the control group, boys had lower scores than girls did 
on academic and social behavior, particularly as rated by teachers. In effect. New Hope raised the 
scores of boys to be approximately equivalent to the scores of girls in control group families. 

Girls in New Hope families manifested a mix of positive and negative responses. De- 
spite the fact that program group girls’ achievement test scores and parent-reported achievement 
were slightly better than those of their control group counterparts, teachers rated New Hope 
girls more negatively on both classroom and social behavior than they did control group girls. In 
turn, program group girls tended to lack engagement with school and school-related activities 
and had lower educational expectations. One possible explanation is that girls in New Hope 
families became more independent or assertive as a result of their parents’ involvement in the 
world of work, but there was no support for this hypothesis in the data (for example, on meas- 
ures of assertiveness and efficacy). A second possible explanation is that New Hope girls may 
have assumed more responsibility at home as the result of their parents’ working more; how- 
ever, there was no evidence that program group girls were assigned more household or child 
care tasks. Third, if parents allocated scarce resources to their sons, daughters in New Hope 
families might have felt resentment; however, if this were the case, one would expect to see im- 
pacts on parents’ ratings of social behavior rather than on teachers’ ratings. The reasons for 
these negative patterns remain unclear, but the findings are nevertheless of concern. If the nega- 
tive impacts on school behavior and involvement continue, they could result in lower educa- 
tional attainment for girls. 

At the five-year evaluation, the oldest children in the sample were adolescents (up to 
age 16). Program impacts on this age group were generally positive. Adolescents in program 
group families had higher levels of school engagement and efficacy and were more likely to 
expect to go to college than adolescents in control group families, and their parents reported 
more effective child management. These findings conflict with a recent meta-analysis of several 
welfare and employment programs, which indicated some negative impacts on adolescents’ 
school progress and deviant behavior.® One important difference between the present New Hope 
sample and those in the meta-analysis is the children’s ages when their parents entered the pro- 
gram. New Hope’s adolescents were in middle childhood (ages 8 to 1 1) when their parents en- 
tered the program. They were still at an age when the child care benefits could be used to pro- 
vide care and supervision while parents were working, whereas children over age 12 were not 
eligible for such benefits. It may be easier to redirect a child’s developmental trajectory in mid- 
dle childhood than in adolescence, in part because parents have more control over the settings in 
which their children spend their time. These findings suggest the importance of child care set- 



®Gennetian et al., 2002a. 
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tings and youth programs in middle childhood as well as adolescence, both as contributors to 
positive youth development and as supports for parental employment. 

One goal of the report is to identify the pathways by which the New Hope intervention 
affected the well-being of families and children. The conceptual model proposed that parental 
employment, greater income, and program benefits would affect children’s everyday experi- 
ences, which, in turn, would influence their development. An experimental design allows one to 
infer that the treatment caused the child outcomes, but it is less informative about which aspects 
of the program were most important. There were modest impacts on family income, characteris- 
tics of parents’ employment, parents’ psychological well-being, and parenting efficacy as well 
as clear impacts on child care and out-of-school activities; but it is difficult, if not impossible, to 
pinpoint the contribution of each factor. 

In fact, there may have been multiple paths of influence. The New Hope offer included 
a “cafeteria” of supports, which enrollees could assemble into a customized package to meet 
their specific needs. The core benefits included a wage supplement, access to community ser- 
vice jobs, and assistance with both child care and health insurance. These concrete supports 
were “wrapped” in a variety of less tangible supports, such as the relationship with a New Hope 
project representative, who worked with the enrollee in administering the core benefits, as well 
as workshops on practical topics and informal get-togethers with other enrollees. Enrollees 
could avail themselves of these “softer” benefits as they saw fit. Many parents in the ethno- 
graphic sample reported that their experiences with New Hope project representatives were very 
positive, particularly in comparison to experiences with welfare offices. The cafeteria model 
means that different participants could have very different experiences in the New Hope Project. 

The diversity of the backgrounds and characteristics of the New Hope population — cou- 
pled with the project’s cafeteria-style set of supports — might imply that different pathways led to 
observed impacts of New Hope for different families. Indeed, the ethnographic researchers ob- 
served varying responses to the New Hope offer as well as different life trajectories across the 
sample. It appeared that New Hope enhanced the overall ability of some program group families 
to find greater stability — to be able to better sustain their family’s daily routine in the face of the 
cascade of problems that so often face working-poor parents. The survey analysis examined im- 
pacts for different ethnic groups and for parents with different initial employment characteristics 
to determine whether the program operated similarly for these groups. Although there were varia- 
tions across groups in the impacts on employment and income, these variations did not translate 
into parallel differential impacts on children. It is possible, and even likely, that these different 
groups used and responded to New Hope in somewhat varied ways. No one pathway accounted 
for effects across all families, but the choices permitted by the program allowed parents to use the 
benefits in ways that fit their overall circumstances and preferences. This model fits the goal of 
allowing parental choice, which was part of the 1996 welfare reform legislation. 
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The results of the evaluation suggest that a program like New Hope may be beneficial 
to large parts of the population who are likely to need welfare. Although the participants in New 
Hope were volunteers for an employment-based incentive program, they were similar in many 
respects to single mothers receiving welfare assistance. The great majority of New Hope par- 
ticipants had received Aid to Families with Dependent Children (AFDC) in the prior year; the 
majority were unemployed; and half of them had earned less than $1,000 in the year prior to 
enrollment in the study. The findings suggest that the components of the New Hope program 
may be useful policy for a wide range of low-income parents and welfare recipients. They sup- 
port the wisdom of recent changes in federal policies that increased the value of the EITC and 
began to extend eligibility for Medicaid and child care subsidies beyond welfare recipients to all 
low-income families. This process is not complete, however, because funds are insufficient to 
meet the needs of all eligible families. It is likely that families and children will benefit fi'om 
continuing the expansion of health and child care subsidies to cover the needs of all low-income 
working families. 

Whatever the reasons, the combination of circumstances brought about by the New 
Hope program led to improved school performance and social behavior, particularly for boys. 
Interventions that create lasting improvements in school performance, motivation, and social 
behavior for boys living in poor families are few and far between. Many children in New Hope 
families are statistically at risk for school failure and other negative consequences. Their family 
incomes are low; most are ethnic minorities; and most are headed by single mothers. If the ex- 
periences provided through New Hope changed young boys’ trajectories toward better school 
performance, more competent social behavior, and less aggression, the chances of school com- 
pletion and socially competent adult development are likely to be increased. A program like 
New Hope may be especially beneficial for children who are in preschool or elementary school 
at its inception. 

Formal child care, extended daycare in schools, and out-of-school activities appear to 
be important paths by which the New Hope program affected children. If this is correct, there 
are clear public policy implications. Public policy can readily increase the availability of center- 
based child care and after-school activities as well as othe r op portunities for supervised, struc- 
tured activities for children and youth — which, in turn, may significantly alter the developmen- 
tal trajectories of boys and girls in low-income families. Such opportunities may be particularly 
important for adolescents, for whom “child care” is no longer appropriate. 

Stepping back, the New Hope experiment establishes the fact that work-based support 
programs can have beneficial effects on children. The annual cost of approximately $5,300 per 
family (not per child) is not trivial. But the benefits of such programs are far fi-om trivial. 
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Appendix A 



Local, State, and National Donors 
for the Pilot Program and the Full Program 
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The New Hope Project 
Appendix Table A.l 

Local, State, and National Donors for New Hope's Pilot Program and Full Program 



Amount and Donor 




$1,000,000 and over 


$25,000 to $49,999 


Helen Bader Foundation 


Bucyrus-Erie 


John D. and Catherine T. MacArthur Foundation 


Robert W. Baird and Company 


Mott Foundation 


Journal Communications 


The Rockefeller Foundation 


Marcus Corporation 


State of Wisconsin 


Marquette Electronics 


U.S. Department of Health and Human Services 


Steigleder Foundation 
United Wisconsin Service 


$250,000 to $999,999 


Universal Foods 


Ameritech / Wisconsin Bell 


University of Michigan 


Annie E. Casey Foundation 


Ford Foundation 


$5,000 to $24,999 


William T. Grant Foundation 


American Express 


Northwestern Mutual Life 


Andersen Consulting 


Wisconsin Energy Corporation 


Arthur Andersen LLP 
Bimschein Foundation 


$100,000 to $249,999 


Emory Clark Foundation 


Firstar 


Patrick and Ann Cudahy Foundation 


Fortis (Time) Insurance 


Dairyland Charitable Trust 


Johnson Controls 


Hamischfeger Industries 


Joyce Foundation 


Helfaer Foundation 


Marshall and Ilsley 


Kohl's Corporation 


Faye McBeath Foundation 


Masterlock 


Milwaukee Foundation 


Judy and David Meissner 


National Institute of Child Health 


Midwest Express 


and Human Development 


North Shore Bank 


WICOR 


Norwest Bank 
Pick Charitable Trust 


$50,000 to $99,999 


Warner Cable Communications 


ANR Pipeline 


Weyenburg Trust 


A.O. Smith 


Banc One 


Under $5,000 


City of Milwaukee 


35 -40 donors 


Harley Davidson 

Mortgage Guaranty Insurance Corporation 
Pollybill Foundation 



SOURCE: The New Hope Project. 
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Sample 

The New Hope Ethnographic Study (NHES) began in spring 1998, during the final year 
of the New Hope experiment, and continued for three years thereafter, through the administra- 
tion of data collection for the five-year survey. The NHES drew a stratified random sample of 
60 families from the 745 families in the Child and Family Study (CFS) sample, with equal rep- 
resentation of both the experimental group and the control group. Of these 60 families, 45 (75 
percent) were enrolled into the NHES. One family dropped out very early in the study, leaving 
44 NHES families in the final sample. 

The demographic profile of the NHES closely matches the profile of the entire CFS 
sample. The NHES sample, just like the CFS sample, is an ethnically diverse, low-income 
group. Slightly more than half the sample were living with a male partner or spouse just prior to 
the start of the study. About a quarter of the families had three or more children, and all families 
at the start of the study had children who were age-eligible (13 or younger) for the child care 
subsidies available from New Hope. 



Fieldwork Methods 

Fieldworkers recruited and then worked with particular families — from one to eight. 
When visiting families, fieldworkers used open-ended interviews to engage parents in conversa- 
tions and descriptions of their lives, concerns, goals, hopes, and everyday activities. Parents 
“told their stories” from their points of view, in their own words, as well as telling the fieldwork 
team about the topics important to the study. The fieldwork team jointly developed a set of do- 
mains and topics to organize these discussions and probed for material relevant to all of them. 
These domains came from the central questions of the New Hope study overall, from previous 
research literature, and from pilot work with families. These domains — the major topics and 
themes which organized the fieldwork visits and field notes — are listed in Appendix Table 
B.l. Fieldworkers also participated in family activities (including meals, shopping, church, 
school, and child care visits and pickups) and talked with the children about their home lives, 
school, and fnends. If parents did not bring up some topics, fieldworkers raised the topics and 
systematically probed for information, ensuring that fieldwork data are as complete as possible 
across all the ethnographic cases. 

After each visit with NHES families, fieldworkers reviewed their notes about the con- 
versations and observations that they had had and wrote visit summaries and more complete 
descriptive field notes. The field notes were based on tape-recordings made during each family 
visit and/or on written notes made during and after the visit. 

The field notes are organized around the topics listed in Appendix Table B.l and are 
stored in and accessed from a FileMaker Pro database specifically developed for this purpose. 
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called EthnoNotes.' Fieldworkers entered their notes into this EthnoNotes system as the study 
progressed, and they rated families’ adaptation and cultural and ecological circumstances using 
this database. EthnoNotes is located on a secure server, so that fieldworkers in Milwaukee or at 
several universities where fieldworkers were based could enter data and read up-to-date field 
notes at any point. In preparing ethnographic research studies and the vignettes and case materi- 
als used in this report, the authors drew systematically on the data stored in EthnoNotes. 

The fieldworkers for the NHES were Conerly Casey (University of California, Los An- 
geles), Nelle Chmielwski (University of Wisconsin, Madison), Victor Espinosa (currently 
Northwestern University), Christina Gibson (Northwestern University; currently Duke Univer- 
sity), Eboni Howard (Northwestern University; currently Chapin Hall Center for Children, Uni- 
versity of Chicago), Katherine Magnuson (Northwestern University; currently Columbia Uni- 
versity), Andrea Robles (University of Wisconsin, Madison), Jennifer Romich (Northwestern 
University; currently University of Washington, Seattle), and Devarati Syam (University of 
Wisconsin, Milwaukee). It is only because of their outstanding collaboration with the families 
in the sample, and their field notes, that the NHES data could be assembled. Other NHES staff 
based at UCLA include Cindy Bemheimer, Helen Davis, Sonya Geis, Eli Lieber, Edward 
Lowe, and Karen Quintiliani. The researchers are most indebted to the families who participated 
in the ethnographic research. 






'Lieber, Weisner, and Presley, Forthcoming, 2003. 
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The New Hope Project 
Appendix Table B.l 

Fieldwork Domains for the New Hope Ethnographic Study: Major Categories and Themes 



Major Categories/Themes 


Minor Categories/Subthemes 


Beliefs/goals/outlook 


Future orientation 


Beliefs/goals/outlook 


Life goals/ambitions 


Beliefs/goals/outlook 


Meaning of work 


Beliefs/goals/outlook 


Political ideology 


Beliefs/goals/outlook 


Relative success standards 


Beliefs/goals/outlook 


Success criteria/definitions 


Beliefs/goals/outlook 


Work attitudes/values 


Children 


Child care as barrier 


Children 


Child care beliefs 


Children 


Child care ideologies 


Children 


Child care issues 


Children 


Child rearing beliefs 


Children 


Child rearing issues 


Children 


Child’s peer network expansion 


Children 


Child’s pride in parents 


Children 


Child’s view of self 


Children 


Gender models/child care 


Children 


General children issues 


Children 


Local resource awareness 


Children 


Safety moves 


Children 


School involvement 


Children 


Sex differences 


Children 


Support from relatives 


Children 


Values in work 



(continued) 
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Appendix Table B.l (continued) 



Major Categories/Themes 


Minor Categories/Subthemes 


Children 


Views of children 


Children 


Work comparisons 


Education 


Education 


Education 


Job skills/credentials 


Education 


Schooling 


Education 


Use of training/education 


Environment control/foresight 




Money/bills/budgeting 


Balances/trade-offs 


Money/bills/budgeting 


Checking/savings account 


Money/bills/budgeting 


Earned Income Tax Credit 


Money/bills/budgeting 


Equity building 


Money/bills/budgeting 


New Hope exit preparation 


Money/bills/budgeting 


Transportation issues 


Money/bills/budgeting 


Unexpected expense cushion 


Nonfamily support 


Client’s view of representative’s role 


Nonfamily support 


Community activism 


Nonfamily support 


Community bridging 


Nonfamily support 


Expanding social networks 


Nonfamily support 


General New Hope participation 


Nonfamily support 


Reference group enhancement 


Nonfamily support 


Relations with case representative 


Nonfamily support 


Social network expansion 


Physical and mental health 


Alcohol/drug issues 


Physical and mental health 


Health/family stress 


Physical and mental health 


Medical care issues 


Physical and mental health 


Mental health 


Presence of family stories 





(continued) 
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Appendix Table B.l (continued) 



Major Categories/Themes 


Minor Categories/Subthemes 


Race 


Employer discrimination 


Race 


Role of ethnicity 


Relationships with partners 


Domestic violence 


Relationships with partners 


Family planning 


Relationships with partners 


Relations with partner 


Relatives 


Family history 


Religious beliefs/practices 


Particular church/faith 


Religious beliefs/practices 


Role of religion/spirituality 


Social networks 




Stability and daily routine 


Daily routine 


Stability and daily routine 


Family pattern adjusment 


Stability and daily routine 


Increased mobility 


Stability and daily routine 


Stability 


Work 


Job as resource 


Work 


Job barriers 


Work 


Job versus career 


Work 


Paths to employment 


Work 


Self-esteem impact 


Work 


Underground economy 


Work 


Work at entry 


Work 


Work experience 


Work 


Work historyWalues 


Work 


Work last year 


Work 


Work readiness 
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Analysis of Nonresponse Bias 
in the Five-Year Survey Sample 
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Most of the impacts shown in this report were estimated from the New Hope survey 
sample — the 561 parents and their children who responded to the five-year survey. Not all par- 
ents and children who were eligible for the five-year survey responded to it. In fact, as shown in 
Figure 2.1, the larger pool of families who were eligible for the Child and Family Study (CFS) 
included 745 sample members who had been randomly assigned to either the program group or 
the control group and whose households contained age-eligible children. Thus, the survey’s re- 
sponse rate was 75 percent — fairly high by conventional standards but low enough, possibly, 
to impart nonresponse bias to the estimates of program effects. 

All CFS sample members had one or more children between the ages of 1 year, 0 
months, and 10 years, 1 1 months, at the time of random assignment. In these households, up to 
two children were selected as focal (that is, a subject of the study), and additional interviews 
were administered to the focal children and to their teachers. From this sample, 539 families (72 
percent) had at least one child who responded to the child and youth surveys.' 

This appendix assesses the extent to which the survey sample is representative of the 
entire CFS sample and, consequently, whether the impacts that were estimated using the survey 
sample may be unbiased. 



Response Rates 

Appendix Table C.l shows response rates — the percentages of eligible sample mem- 
bers in the CFS sample who responded to the five-year surveys of parents, children and youth, 
and teachers. Overall, 75 percent of parents and 72 percent of children in the CFS sample pro- 
vided at least some responses to the five-year surveys. Some 63 percent of all children who re- 
sponded had at least one teacher who responded to the five-year teacher survey.^ 

Appendix Table C. 1 also presents response rates by research group. It is particularly 
important in a random assignment design that response rates be similar for the program group 
and the control group. Although response rates for the program and control groups differed 
somewhat between the two samples, none of these differences were statistically significant at 
conventional levels. 



'since up to two children per household could be focal children, the total number of children who were in- 
terviewed is higher — 840 in all. The figures for children presented in this appendix are based on the number 
of families in which at least one child responded to the survey. 

^Most nonresponse among teachers resulted from the inability to locate the right teacher rather than from 
nonresponse by the teachers or refusal by the parents to give permission. In cases where the child had multiple 
teachers, more than one teacher per child could be interviewed. Altogether, 674 teachers responded to the sur- 
vey. The response rate for teachers is determined on the basis of the number of child-respondents who had at 
least one teacher respond on their behalf — regardless of whether more than one teacher responded. 
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The New Hope Project 
Appendix Table C.l 
Five-Year Survey Response Rates 



Respondent 


Sample 

Size 


CFS 

Sample 

(%) 


Program 

Group 

(%) 


Control 

Group 

(%) 


Parent responded 


745 


75.3 


77.1 


73.5 


Any child responded 


745 


72.3 


74.2 


70.6 


Any teacher responded 


840 


63.2 


63.0 


64.7 



SOURCES: MDRC calculations using data from the New Hope parent, child, youth, and teacher five- 
year surveys. 



NOTE: Two-tailed t-tests applied to differences between the program and control groups revealed that 
none were statistically significant at the .10 level. 



Comparison of Baseline Characteristics 

Although three-quarters of the CFS parents responded to the five year survey, there is 
still a real chance that response bias exists. Appendix Table C.2 examines this issue more di- 
rectly by showing selected baseline characteristics for the entire CFS sample and for respon- 
dents and nonrespondents to the five-year survey. Asterisks in the rightmost column denote in- 
stances in which the characteristics of respondents and nonrespondents differed significantly. 

A comparison of the second and third columns shows that survey respondents and 
nonrespondents were similar in most respects, although some statistically significant differences 
were found. Compared with nonrespondents, survey respondents were significantly less likely 
to be male, more likely to have ever worked full time prior to random assignment, and more 
likely to have lived as a child in a household that received- Aid to Families with Dependent 
Children (AFDC). All the estimates of program impacts control for baseline differences in the 
characteristics listed in Appendix Table C.2, which should eliminate most bias that might be 
correlated with these measured characteristics. The possibility of bias associated with unmeas- 
ured characteristics that are not correlated with these baseline measures remains. 
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The New Hope Project 
Appendix Table C.2 



Comparison of the Baseline Characteristics of the CFS Sample 



Characteristic (%) 


Entire 

CFS 

Sample 


Respondents to 
the Five-Year 
Survey 


Nonrespondents to 
the Five-Year 
Survey 


Male 


10.2 


8.6 


15.2 *** 


Northside target area 


48.6 


49.6 


45.7 


Age under 25 


31.7 


32.1 


30.4 


Age 25-34 


49.0 


48.8 


49.5 


African-American, non-Hispanic 


55.0 


55.6 


53.3 


Hispanic 


29.3 


28.3 


32.1 


Household with children and one adult 


83.5 


84.1 


81.5 


Household with three or more children 


45.9 


47.2 


41.8 


Youngest child age 2 or younger 


48.3 


48.0 


49.5 


Ever worked full time 


83.4 


85.0 


78.3 ♦♦ 


Received public assistance at baseline 


80.7 


81.1 


79.3 


Had high school diploma or GED 


59.5 


60.6 


56.0 


Had access to a car 


44.1 


43.9 


44.6 


In AFDC household as a child 


46.7 


48.9 


39.6 ** 


Sample size 


745 


561 


184 



SOURCES: MDRC calculations using data from the New Hope Background Information Form (BIF) and 
the five-year parent survey. 



NOTE: Statistical tests of difference were conducted only between the five-year survey sample and the 
nonrespondents. Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 
percent. 



Comparison of Impact Estimates from Administrative Records 

Another way to assess whether analyses using the five-year survey sample are likely to 
be biased is to compare impacts for the entire CFS sample and for survey respondents and non- 
respondents on outcomes observed for all groups. This is the case with the administrative re- 
cords data on earnings, benefit receipt, and income,^ since data were gathered from all families 
in the CFS sample, including both respondents and nonrespondents to the five-year survey. 
These results are presented in Appendix Table C.3. 



^Income includes earnings, AFDC payments, food stamps, supplement payments, and estimated Earned 
Income Tax Credits (EITCs). 



The New Hope Project 
Appendix Table C.3 

Comparison of Impacts on the Earnings, Income, and Employment of the Entire 
CFS Sample and the Survey Sample 



Outcome 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value for 
Difference 
Across 
Samples^ 


Entire CFS samole 
Average annual earnings ($) 


Years 1-2 


7,659 


6,903 


756 ** 




Years 3-5 


10,453 


9,895 


557 




Average annual income ($) 


Years 1-2 


14,073 


12,924 


1,150 *** 




Years 3-5 


14,018 


13,319 


699 




Average annual AFDC payment ($) 


Years 1-2 


2,647 


2,796 


-149 




Years 3-5 


704 


750 


-47 




Average quarters employed (%) 


Years 1-2 


3.0 


2.7 


0.3 *** 




Years 3-5 


2.9 


2.8 


0.1 




Sample size 


366 


379 







Respondents to the five-year survey 



Average annual earnings (S) 



Years 1-2 


8,250 


7,283 


967 ** 


0.17 


Years 3-5 


11,420 


10,705 


715 


0.35 


Average annual income ($) 


Years 1-2 


14,794 


13,392 


1,401 *** 


0.23 


Years 3-5 


15,428 


14,263 


1,164 * 


0.10 


Average annual AFDC payment (S) 


Years 1-2 


2,678 


2,769 


-90 


0.92 


Years 3-5 


840 


761 


79 


0.04 


Average quarters employed (%) 


Years 1-2 


3.1 


2.8 


0.4 *** 


0.30 


Years 3-5 


3.2 


3.0 


0.1 


0.49 


Sample size 


282 


279 







(continued) 
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Appendix Table C.3 (continued) 











P- Value for 










Difference 




Program 


Control 


Difference 


Across 


Outcome 


Group 


Group 


(Impact) 


Samples^ 



Nonrespondents to the five-year survey 



Average annual earnings (S) 



Years 1-2 


5,662 


5,929 


-267 


Years 3-5 


7,197 


7,721 


-524 


Average annual income ($) 


Years 1-2 


11,753 


11,618 


135 


Years 3-5 


9,492 


10,661 


-1,169 


Average annual AFDC payment ($) 


Years 1-2 


2,653 


2,779 


-126 


Years 3-5 


334 


673 


-339 ♦♦ 


Average quarters employed (%) 


Years 1-2 


2.5 


2.4 


0.1 


Years 3-5 


2.2 


2.2 


0.0 


Sample size 


84 


100 





SOURCES: MDRC calculations using data from the New Hope five-year parent survey and 
Wisconsin unemployment insurance (UI) records. 



NOTES: Two tailed t-tests were applied to differences between the program and control groups and to 
the differences between the CFS sample and the five-year survey sample. Statistical significance 
levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

“A statistical test was conducted to measure whether impacts presented for different groups in 
this table were significantly different from one another. This p- value represents the probability that 
apparent variation in impacts across different panels in the table is simply the result of random chance. 
If this probability is less than 10 percent, the variation in impacts is considered statistically significant. 



The first two rows of each panel present impacts on average annual earnings. A direct 
comparison of the respondent and nonrespondent samples shows a number of differences in 
impact estimates. This is clearly shown in the rightmost column of the panel for respondents, 
which presents the results of a test that assesses whether the variation in impacts across the 
respondent and nonrespondent samples are statistically significant. In Years 1-2, for example. 
New Hope increased earnings by nearly $1,000 for the survey respondent sample, but it de- 



creased earnings by $267 for the nonrespondent sample. The significance level of this differ- 
ence was p = .17. The only statistically significant difference in impacts between respondent 
and nonrespondent samples was in the case of AFDC receipt in Years 3-5, which averaged 
$79 more for program group families than control group families who responded to the five- 
year survey but $339 less for program group families than control group families who chose 
not to respond to the survey. All in all, the administrative records data show statistically indis- 
tinguishable impacts for the survey respondent and nonrespondent samples. 

Comparison of Program and Control Groups in Different Samples 

To further assess whether treatment/control status was related to survey response, a re- 
gression of response status on treatment/control status — plus the demographic characteristics 
gathered at random assignment — was performed. As shown in Appendix Table C.4, treat- 
ment/control status was not a significant predictor of survey response for any of the samples 
used in the analyses. 
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Appendix Table C.4 

Treatment/Control Status as a Predictor of Survey Response 
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NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 
“Child's gender and age are excluded from the first two regressions since the unit of analysis is the family. 
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PARENTS’ CONTEXTS 



Parents’ Well-Being 

Material Well-Being 

Material Hardship. Parents responded to a summary measure of six items assessing 
whether they or their family members in the past 12 months had experienced any of a number 
of hardships (such as being without phone service or being unable to pay the full amount of rent 
or mortgage). Parents responded on a two-point scale (0 = “no,” 1 = “yes”). The internal consis- 
tency for this scale was .69. 

Financial Well-Being. Using a five-point scale (1 = “not at all true,” 5 = “very true”), 
parents were asked to rate their level of financial well-being (for example, “I worry about hav- 
ing enough money in the future”). The internal consistency for this scale was .08. Deleting any 
of the items did not improve the scale’s reliability. 

Financial Worry. Several items from the interview assessed parents’ feelings of worry 
about financial issues. Using a five-point scale (1 = “not at all,” 5 = “a great deal”), respondents 
indicated how much they worried about paying the bills, gaining employment, having medical 
coverage, having money for food, and being able to afford adequate housing. Responses to the 
five items were averaged to compute a “financial worry”.score. An internal consistency level of 
.90 suggests that the five items measure a unified construct. 

Residential Well-Being 

Neighborhood Resources. Five items assessed parents’ satisfaction with aspects of 
their neighborhood, including “places to shop for things” and “level of children’s safety.” Par- 
ents indicated their satisfaction by responding on a five-point scale (1 = “very unhappy,” 5 = 
“very happy”). The internal consistency for this scale was .72. 

Collective Efficacy. Parents were asked about their level of confidence that their 
neighbors would respond to such neighborhood threats as “someone breaking into your home in 
plain sight.” Parents answered on a five-point scale (1 = “very unlikely,” 5 = “very likely”). The 
internal consistency for this scale was .85. 

Parents’ Coping Strategies and Instrumental Behaviors 

Sustainability. Based on parents’ responses during ethnographic fieldwork, five items 
were developed to assess such aspects of family sustainability as having enough resources and 
being able to manage and juggle life’s daily routines. Parents were asked, for instance, “Are you 
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more like the people: who feel they have enough resources or those who feel they just never 
have enough?” Then, for the same question, they responded to the question “Is that: sort of true 
for you or very true for you?” This format resulted in four response options for each of the five 
items. The alpha for this scale was .71. 

Parents’ Psychosocial Well-Being 

Depression. Parents’ experience of depression was assessed using the Center for Epi- 
demiological Studies-Depression (CES-D) scale.' The 20-item self-report scale is intended as a 
screening instrument and has been used in many large-scale projects involving low-income 
adults similar to those in the New Hope study. Parent’s responded to nine items regarding their 
feelings and behaviors in the past week (for example, “I felt lonely,” “I had crying spells”), us- 
ing a four-point scale (1 = “rarely or none [less than 1 day],” 4 = “most or all [5 to 7 days]”). 
The alpha for this scale was .82. 

Hope. The State Hope Scale^ was used to assess parents’ sense of hope, in terms of 
agency (belief in one’s capacity to initiate and sustain actions) and pathways (belief in one’s 
capacity to generate routes to achieve goals). Using a four-point scale ranging from “strongly 
disagree” to “strongly agree,” respondents indicated their level of agreement with six items (for 
example, “I am meeting the goals I set for myself’). A reliability coefficient of .85 suggests that 
the State Hope Scale is a reliable measure of the related constructs of agency and pathways. 

Substance Use/Abuse. Parents were asked five items concerning the frequency of their 
use of substances (including tobacco, alcohol, and other drugs) in the past month and two items 
about the frequency of indicators of substance abuse in the past month (for example, people 
complaining about their use of dmgs or alcohol). Using a five-point scale ranging from “no 
days” to “almost every day,” respondents indicated their usage of tobacco, alcohol, and other 
drugs during the past 30 days An alpha coefficient of .54 was obtained. When the item “smoked 
cigarettes” was removed from the scale, the reliability increased to .72. 



'Radloff, 1977. 
^Snyder et al., 1996. 
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CHILDREN’S CONTEXTS 



Parenting 

Effective Child Management 

Parental Problems with Control. A five-item consistency scale from the Canadian 
evaluation of the Self-Sufficiency Project (SSP)^ was used to measure a dimension of parenting 
termed “control.” Using a six-point scale ranging from “never” to “all of the time,” parents were 
asked to indicate the frequency of five discipline events (for example, how often the child ig- 
nored the parent’s punishment). Based on pilot-testing and item analysis, the five items were 
selected from a larger set that was used in SSP. The scale had a reliability coefficient of .80, 
indicating sufficient internal consistency. 

Parental Discipline. Nine items were taken from the SSP evaluation"' to assess how often 
in the past week parents had used disciplinary action with their child (for example, grounding, 
spanking, taking away privileges). A four-point response scale was used, ranging from “never” to 
“4 or more times.” An internal consistency coefficient of .83 was obtained for the scale. 

Parenting Stress. Two scales that were used in the New Chance evaluation^ were used 
to assess the degree of stress or aggravation perceived by the parent in relation to interactions 
with the child. The first scale, a measure of general parenting stress, consisted of three items 
concerning negative feelings about the parental role (for example, “I feel trapped by my respon- 
sibilities as a parent”). The second scale consisted of five items designed to measure stress spe- 
cifically associated with the target child (for example, “My child seems to be much harder to 
care for than most”). Both sets of items used a five-point response scale ranging from “not at all 
true” to “very true.” The internal consistency coefficients for general parenting stress and child- 
specific parenting stress were .61 and .79, respectively. The two measures were slightly corre- 
lated (r = .35), suggesting that different types of stress may be experienced by parents in relation 
to child-rearing. 

Positive Youth-Parent Relations 

Child- and Youth-Reported Positive Parent-Child Relations. The Child Evaluation 
of Relationship with Mother/Caregiver measure was developed as part of a study of low-income 



^Statistics Canada, 1995. 
‘‘Statistics Canada, 1995. 
^Quint, Bos, and Polit, 1997. 
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African-American families.® Children aged 6 to 12 at the time of the New Hope survey indi- 
cated on a five-point scale (1 = “not at all true,” 5 = “very true”) how true 19 statements were 
about their parent, their relations with the parent, and interactions with the parent. Items were 
adapted from a rating instrument developed by Swanson’ and revised by McLoyd and col- 
leagues.® Two subscales were derived, one that comprised 12 items assessing perceived positive 
parent-child relations (for example, “Your parent spends a lot of time talking with you”) and 
another that comprised 7 items tapping perceived negative parent-child relations (for example, 
“It is hard to be pleasant and happy around your parent”). Within each of the two subscales, 
items were summed to create a total score, with higher scores indicating a more positive or a 
more negative quality, respectively. 

McLoyd and colleagues’ reported high levels of internal consistency for the positive 
and negative relations subscales: The alphas were .91 and .81, respectively. In New Hope’s 
five-year survey sample of 9- to 12-year-olds, internal consistency coefficients were .90 for Per- 
ceived Positive Relations and .66 for Perceived Negative Relations. Children were more consis- 
tent in their responses to items about positive interactions with their parent than items about 
negative interactions. 

Youth-Reported Parental Acceptance and Involvement. Children aged 9 to 16 re- 
ported the degree to which their parents made them feel accepted and were involved in their 
lives, by responding to nine items from the “acceptance/involvement” subscales of the Authori- 
tative Parenting Measure.” A four-point response scale ranging from “strongly disagree” to 
“strongly agree” was used, and a mean score was calculated for each subscale. An example 
from this scale is “Your parent says you shouldn’t argue with adults.” The internal consistency 
for this subscale was .69. 

Youth-Reported Parental Monitoring. Children aged 9 to 16 reported on their par- 
ents’ monitoring by assessing the extent to which their parents knew about their activities and 
their fnends (for example, “Does your parent know what you are doing after school?”). Chil- 
dren answered on a four-point scale (1 = “strongly disagree,” 4 = “strongly agree”). The alpha 
for this scale was .71. 



®McLoyd, Jayaratne, Ceballo, and Borquez, 1994. 
’Swanson, 1950. 

®McLoyd, Jayaratne, Ceballo, and Borquez, 1994. 
’McLoyd, Jayaratne, Ceballo, and Borquez, 1994. 
”Steinberg, Lambom, Dombusch, and Darling, 1992. 



Negative Youth-Parent Relations 

Child- and Youth-Reported Negative Parent-Child Relations. See the section above 
entitled “Child- and Youth-Reported Positive Parent-Child Relations.” 

Youth-Reported Parental Autonomy Granting. Children aged 9 to 16 reported on 
the degree to which they felt that their parents granted them autonomy, by responding to nine 
items from the “psychological autonomy granting” subscale of the Authoritative Parenting 
Measure." A four-point response scale ranging from “strongly disagree” to “strongly agree” 
was used, and a mean score was calculated for each subscale. An example from the autonomy 
granting subscale is “When you get a poor grade in school your parent encourages you to try 
harder.” The internal consistency for this subscale was .62. 

Warm and Structured Parenting 

Self-Reported Parental Warmth. Parents reported on their displayed warmth during 
interactions with their children using a three-item “warmth scale” from the Canadian evaluation 
of the Self-Sufficiency Project." Using a six-point response scale ranging from “never” to 
“many times each day,” parents indicated the frequency of their praise, focused attention, and 
special activities involving their child. The three items were averaged to compute a scale score. 
The internal consistency for the three-item scale was .82, indicating that parents responded simi- 
larly across the items. 

Observer-Reported Parental Warmth. Parental warmth was also assessed by observ- 
ers using two items from the HOME measure: the extent to which parents “conveyed positive 
feelings about their children” and “spontaneously praised or talked about their children’s good 
qualities or behavior.”" The reliability coefficient for these items was .92. 

Regularity of Family Routines. Parents were asked six questions assessing the fre- 
quency with which their family participated in regular activities, including how often “children 
did homework around the same time at night” and whether family members “ate dinner or sup- 
per together most nights of the week.” The internal consistency for this scale was .80. 

Parenting Behavior 

Parent-Reported Monitoring. To measure parents’ monitoring of children’s activities, 
six items w'ere taken from the five-year follow-up to parent and child assessments in the Job Op- 
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Steinberg, Lambom, Dombusch, and Darling, 1992 
'^Statistics Canada, 1995. 

'^Caldwell and Bradley, 1984. 



portunities and Basic Skills Training (JOBS) program.’'' Items assessed parents’ knowledge about 
their child’s TV viewing, who their child was with when away from home, where their child was 
when away from home, and how many of their child’s friends they knew by first or last name. For 
children aged 12 and older, parents were also asked three items about curfews. A sbc-point re- 
sponse scale ranging from “never” to “always” was used. The monitoring scale had an internal 
consistency level of .86, suggesting that parents were consistent in their responses to its items. 

Prevention-of-Harm Strategies. Parents were asked the degree to which they used dif- 
ferent parenting strategies to prevent their children from getting into trouble (for example, get- 
ting their child involved in organized activities, enforcing rules). Parents were asked six ques- 
tions on a five-point scale ranging from “almost never” to “very often.” The reliability coeffi- 
cient for this scale was .73. 



Children’s Activities 

Children’s and Parents’ Reports of Activities. Both parents and children were asked 
about children’s use of time and participation in stmctured, out-of-school activities during the 
school year and the summer. Parents and children responded on a five-point scale (1 = “never,” 
5 = “about every day”) to report how frequently the children participated in such activities as 
organized sports, clubs, and lessons. The alpha for parents reports of children’s activities during 
the school year was .53, and the alpha was .54 for the summer. For children’s reports of activi- 
ties, the alpha was .58 for the school year and .63 for the summer. 

Parental Approval of Children’s Activities. Parents responded on a five-point scale 
(1 = “not at all,” 5 = “a lot”) to report the extent to which they considered participation in five 
activities to be beneficial or harmful for their child. The activities were lessons, sports with a 
coach, clubs and youth groups, recreation centers, and working for pay. The reliability for this 
scale was .84. 

CHILDREN’S OUTCOMES 
Education 

All measures of achievement were administered to the entire age range studied. 



‘‘'U.S. Department of Health and Human Services Web site; http://www.acf.dhhs.gov/programs/JOBS. 



standardized Achievement Test Scores 



To assess reading and mathematical competencies, children completed four individually 
administered scales from the Woodcock-Johnson Achievement Battery.'^ Two of these — Let- 
ter-Word Identification and Passage Comprehension — measure reading skills; the sum of these 
two is the Broad Reading score. The other two scales — Applied Problems and Calculation — 
measure mathematics skills; the sum of these two is the Broad Math score. The total score is the 
sum of all four scales. The Woodcock-Johnson was selected because its normative sample is 
large and representative and because the sample includes children from diverse ethnic groups 
and diverse types of schooling. The standard score for each scale is obtained by comparing the 
child’s score with norms for his or her chronological age group. The mean standard score for 
the population as a whole is 100; the standard deviation is 15. Standard reliabilities for the 
Broad Math and Broad Reading scores were each .95. 

Parents’ Ratings of Achievement 

Based on their knowledge of recent report cards, parents evaluated their child’s per- 
formance in reading, mathematics, and written work, using a five-point scale ranging from “not 
at all well” to “very well.” The alpha for this scale was .87. 

Teachers’ Ratings of Achievement 

Mock Report Card. Teachers used a mock report card to rate children’s performance 
in reading, oral and written language, math, social studies, and science. Teachers responded on a 
five-point scale (1 = “below average,” 5 = “ excellent”). This measure was adapted from one 
used in the NICHD Study of Early Child Care and Youth Development.'® The alpha for this 
scale was .90. 

Academic Subscale. Teachers responded to the 10-item academic subscale of the So- 
cial Skills Rating System.'^ They used a five-point scale (1 = “lowest 10 percent of class,” 5 = 
“highest 10 percent of class”) to compare children’s performance with others in the same class- 
room; they assessed reading skills, math skills, intellectual fiinctioning, motivation, oral com- 
munication, classroom behavior, and parental encouragement. The alpha for this scale was .94. 

Classroom Behavior Scale. Teachers completed the Classroom Behavior Scale, which 
contains items concerning children’s study skills, conformity to classroom rules and routines, 
ability to work and complete tasks independently, and ability to make transitions without be- 



' ® Woodcock andJohnson, 1990. 
'®NICHD, 2002. 

'^Gresham and Elliot, 1990. 
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coming distracted.'* Teachers responded on a five point scale (1 = “almost never,” 5 = “almost 
always”). The alpha for this scale was .97. 

Teachers’ Expectations for a Child. Three items measured teachers’ expectations for 
the focal child’s educational attainment. Using a five-point scale ranging from “not at all” to 
“very,” teachers indicated how sure they were that the child would finish high school, go to col- 
lege, and finish college. These three items were summed to produce one score. The combined 
alpha for the items was .92. 



Children’s Motivations and Beliefs 

Competency Beliefs 

Competency Beliefs and Task Values for Reading and Math. Items were adapted 
from the Self and Task Perception Questionnaire” and contained questions assessing children’s 
self-concept of ability, expectations for success, extrinsic and intrinsic utility value, and attain- 
ment value regarding English/reading and math (for example, “How good at English are you?” 
“How useful is what you learn in math?”). The alphas for the English/reading and the math 
items were .82 and .85, respectively. 

Children’s Efficacy. Children’s hope was assessed using the Children’s Hope Scale.“ 
The scale contains two subscales — pathways and agency. — each of which includes three 
items (for example, “I think I’m doing pretty well”; “Even when others want to quit, I know I 
can find ways to solve the problem”). Children rated the items using a six-point response scale 
(1 = “ none of the time,” 6 = “all of the time”). For the entire scale, the reliability was .81. 

School Engagement. Children aged 9 to 1 6 reported their perceptions of the extent to 
which they felt engaged at school, as assessed through five items (for example, “You feel close 
to others at your school”). Children responded on a five-point scale ranging from “not true at 
all” to “always tme for you.” Items were adapted from the Adolescent Health Study.^' The al- 
pha for this scale was .84. 

Values for the Future 

Children’s Attitudes About Work. Children’s attitudes about work (for example, 
“You expect work to be a very central part of your life”) were obtained using five items taken 

'*Wright and Huston, 1995. 

”Eccles et al., 1983; Ecdes and Wigfield, 1995. 

^“Snyder et al., 1996. 

^'http://www.qjc.unc.edu/projects/addhealth, 1998. 
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from the Monitoring the Future report.^^ Children answered on a five-point scale ranging from 
“strongly disagree” to “strongly agree.” The alpha for this scale was .46. 

Importance of Future Achievements. Children were asked seven general questions 
about the importance of future achievements, using items adapted from Flanagan and col- 
leagues.^^ Children used a five-point scale (1 = “not at all important,” 5 = “very important”) to 
answer such questions as “How important to you is being close to your family?” The alpha for 
this scale was .43. 



Children’s Social Behavior 

Positive Behavior 

The Positive Behavior Scale was developed for the New Chance survey,^'' a study of 
more than 2,000 low-income mothers and their children. A parallel version for teachers contains 
similar or identical items. Its 25 items can be divided into three subscales: compliance/self- 
control (for example, “thinks before he/she acts,” “usually does what I tell him/her”); social 
competence and sensitivity (for example, “gets along well with other children,” “shows concern 
for other people’s feelings”);and autonomy (for example, “tries to do things for hirn/herself,” “is 
self-reliant”). The parent or teacher responds on a five-point scale ranging from “never” to “all 
of the time.” 

The Positive Behavior Scale was chosen for this study instead of a similar set of items 
from the Social Skills Rating System because it was judged by the investigators and community 
representatives in Milwaukee as more appropriate for the populations being studied and because 
it had been standardized on a multiethnic sample of mothers who had low incomes. Items for 
adolescents were adapted to be age-appropriate. For the New Chance sample, the internal con- 
sistency of the total score was .94 for parents’ ratings (internal consistencies for the subscales 
ranged from .77 to .88). In New Hope’s CFS sample, the internal consistency for the total score 
was .91 for parents’ ratings and .96 for teachers’ ratings. The consistencies within subscales 
ranged from .71 to .86 for parents and from .81 to .92 for teachers; that is, both parents and 
teachers were fairly consistent in their descriptions of these qualities in children. 





^^http://www.monitoringthefuture.org, 1995. 
^^Flanagan et al., 1998. 

^‘*Quint, Bos, and Polit, 1997. 



Problem Behavior 



To assess children’s negative social behavior, the Problem Behavior Scale from the So- 
cial Skills Rating System was administered to both parents and teachers.^^ Parents received two 
components: externalizing problems and internalizing problems. Externalizing problems in- 
clude aggression and lack of behavior control (for example, “is aggressive toward people or ob- 
jects,” “has temper tantrums”). Internalizing problems include social withdrawal and excessive 
fearfulness (for example, “appears lonely,” “acts sad or depressed”). Teachers completed the 
externalizing and internalizing items as well as a hyperactivity component (for example, “is eas- 
ily distracted,” “disturbs ongoing activities”). Teachers also reported how often they had to dis- 
cipline the child for misbehavior. The internal consistencies for parents’ ratings were .77 for the 
total score and .61 to .81 for the components. Internal consistencies for teachers’ ratings ranged 
from .78 to .92. 

Social Relationships 

Perceived Quality of Peer Relationships and Friendships. The Loneliness and Social 
Dissatisfaction Questionnaire measures the child’s perceptions of peer relations and fnend- 
ships.^® It contains 16 items that loaded on one factor in the standardization sample of 200 chil- 
dren in grades 3 through 6 (for example, “It’s hard for me to make new fnends”). Children 
aged 6 to 8 answered on a three-point scale, and those aged 9 to 12 answered on a five-point 
scale (1 = “always true,” 5 = “not true at all”). The internal consistency for this scale was .89. 
Scores for 6- to 8-year-olds were prorated to be equivalent to the older children’s scores, by 
multiplying each item by 5/3. 

Intent Attribution. The Intent Attributions and Feelings of Distress Measure^’ is a hy- 
pothetical provocation instrument used to assess children’s intent attributions and feelings of 
distress when in ambiguous relational and instrumental provocation situations. The measure 
consists of four stories and two questions about each. Children’s choices reflect their percep- 
tions of the actor in the story as having either “hostile” or “benign” intent. For intent attribu- 
tions, the alpha was .80. 

Peer Group Conventional Behaviors. This instrument from the Pittsburgh Youth 
Study^® assesses conventional activities of fnends and includes nine items (for example, “In the 
past 12 months how many of your close fnends have been involved in school sports?”). Chil- 



^^Gresham and Elliot, 1990. 

Asher and Wheeler, 1985; Cassidy and Asher, 1992. 
^^Crick and Dodge, 1996. 
^*http://ojjdp.ncjrs.org/ccd/pittsburgh.html. 
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dren responded to the items using a five-point scale ranging from “none of them” to “all of 
them.” The alpha for this scale was .77. 

Risky Behavior 

Delinquent Behavior. Adapted from LeBlanc and Tremblay’s 27-item measure assess- 
ing adolescents’ self-reported deviant behavior, for New Hope this measure was shortened to 1 5 
items and used modified wording.^’ Children responded on a five-point scale ranging from 
“never” to “5 or more times” to answer such questions as “During the past 12 months did you 
take part in a gang fight?” There are four subscales: fighting, stealing, vandalism, and dmgs. 
The total scale has been validated on more than 6,000 teenagers in Quebec.^ The alpha for this 
scale was .66. 

Peer Group Delinquent Activities. Adapted from LeBlanc and Tremblay’s 27-item 
measure of adolescents’ reports of peers’ deviant behavior, for New Hope this measure was 
shortened to 15 items and used modified wording.^' Children responded on a five-point scale 
ranging from “none of them” to “all of them” to answer such questions as “In the past 12 
months how many of your close friends had a fistfight with another person?” There are four 
subscales: fighting, stealing, vandalism, and dmgs. The alpha for this scale was 69. 



^’LeBlanc and Tremblay, 1988. 
^“LeBlanc and Tremblay, 1988. 
^'LeBlanc and Tremblay, 1988. 



- 212 - 



248 



Appendix E 

Supplementary Tables to Chapters 4, 5, and 6 



i 



249 



o 

ERIC 



The New Hope Project 
Appendix Table E.4.1 

Impacts on Parental Well-Being, by Number of Potential Barriers to Employment 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size“ 


P- Value for 
Difference 
Across 

Barrier Groups^ 


No potential barriers 
















Material well-being 
















Family income: survey report ($) 


23,554 


23,891 


-337 


0.919 


-1.4 


-0.02 


0.641 


Standard of living 


3.8 


3.7 


0.0 


0.867 


0.8 


0.03 


0.685 


Material hardship 


0.2 


0.2 


0.0 


0.791 


-5.3 


-0.04 


0.172 


Food security 


3.4 


3.4 


0.0 


0.899 


-0.5 


-0.02 


0.580 


Financial well-being 


16.4 


16.8 


-0.5 


0.559 


-2.7 


-0.09 


0.360 


Financial health 


0.5 


0.5 


0.0 


0.907 


-0.9 


-0.02 


0.554 


Financial support to others 


14.5 


9.6 


4.9 


0.379 


50.8 


0.16 


0.087 t 


Financial worry 


2.4 


2.4 


0.0 


0.974 


-0.3 


-0.01 


0.193 


Residential well-being 
















Number of moves in past 3 years 


1.2 


0.9 


0.2 


0.253 


26.9 


0.16 


0.590 


Satisfaction with housing 


3.0 


3.1 


-0.1 


0.662 


-2.1 


-0.07 


0.268 


Bad housing conditions 


1.1 


1.1 


0.0 


0.682 


-1.1 


-0.06 


0.771 


Crowded housing 


1.6 


1.6 


0.0 


0.966 


0.3 


0.01 


0.776 


Neighborhood good to raise children 


3.0 


2.9 


0.1 


0.617 


2.9 


0.08 


0.793 


Neighborhood resources 


3.6 


3.5 


0.0 


0.754 


1.4 


0.05 


0.409 


Collective efficacy 


4.0 


3.9 


0.1 


0.596 


2.8 


0.09 


0.871 


Housing safety: observer 


1.4 


1.4 


0.0 


0.781 


1.1 


0.05 


0.518 


Neighborhood problems: observer 


9.4 


8.9 


0.5 


0.369 


5.2 


0.15 


0.317 


Physical well-being 
















Physical health 


3.6 


3.4 


0.2 


0.181 


7.2 


0.21 


0.648 


Health condition hinders work (%) 


19.1 


21.4 


-2.4 


0.727 


-11.0 


-0.06 


0.783 


Hospitalizations in last year 


0.2 


0.2 


0.0 


0.873 


-9.2 


-0.01 


0.100 t 


Substance use/abuse 


1.3 


1.3 


0.0 


0.859 


-0.9 


-0.03 


0.240 


Coping strategies and instrumental 
















behaviors 
















Sustainability 


3.2 


3.0 


0.2 


0.181 


5.3 


0.20 


0.343 


Pursuing any goals (%) 


78.5 


83.5 


-5.0 


0.454 


-6.0 


-0.13 


0.772 


Achieving any goals (%) 


75.0 


62.2 


12.7 ♦ 


0.096 


20.4 


0.28 


0.214 


Awareness of helping resources 


2.0 


1.9 


0.1 ♦ 


0.095 


6.2 


0.27 


0.938 


Community involvement 


1.4 


1.4 


0.0 


0.822 


-0.8 


-0.04 


0.395 


Married and living with spouse (%) 


21.1 


14.0 


7.1 


0.229 


50.9 


0.17 


0.587 


Not married and living with partner (%) 


30.1 


25.4 


4.7 


0.560 


18.6 


0.11 


0.561 


Discouragement of working 


2.7 


2.5 


0.2 


0.278 


8.2 


0.19 


0.189 


Practical supports for working 


2.1 


2.1 


0.0 


0.790 


1.4 


0.04 


0.981 


Used EITC last year (%) 


67.6 


62.6 


5.0 


0.535 


7.9 


0.11 


0.526 


Aware of EITC last year (%) 


94.7 


95.7 


-1.0 


0.779 


-1.1 


-0.04 


0.242 
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Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


P-Value for 
Difference 
Effect Across 

Size^ Barrier Groups^ 


Psychosocial well-being 














0.118 


General life stress 


2.4 


2.5 


-0.1 


0.340 


-5.2 


-0.15 


Time pressure 


3.2 


3.5 


-0.3 


0.118 


-7.3 


-0.24 


0.122 


Job quality 


0.7 


0.7 


0.0 


0.671 


-4.0 


-0.07 


0.376 


Difficult life circumstances 


0.2 


0.2 


0.0 


0.818 


3.0 


0.03 


0.926 


Depression 


14.1 


16.7 


-2.6 


0.163 


-15.6 


-0.24 


0.830 


Religiosity 


3.7 


3.6 


0.1 


0.384 


2.6 


0.13 


0.478 


Hope 


3.0 


3.1 


-0.1 


0.297 


-2.8 


-0.16 


0.139 


Influence of close others 


3.8 


3.7 


0.1 


0.316 


3.4 


0.16 


0.124 


Availability of supportive others 


3.2 


3.1 


0.1 


0.621 


2.2 


0.08 


0.754 


Perception of life now versus 
















5 years ago 


4.3 


4.3 


0.0 


0.974 


0.1 


0.01 


0.881 


Sample size 


129 


101 












One potential barrier 
















Material well-being 
















Family income: survey report ($) 


23,495 


21,325 


2,170 


0.310 


10.2 


0.13 




Standard of living 


3.9 


3.8 


0.1 


0.463 


2.6 


0.09 




Material hardship 


0.2 


0.2 


0.0 


0.241 


-19.3 


-0.16 




Food security 


3.5 


3.5 


0.0 


0.989 


0.0 


0.00 




Financial well-being 


16.4 


16.2 


0.2 


0.707 


1.5 


0.05 




Financial health 


0.5 


0.5 


0.0 


0.890 


0.9 


0.02 




Financial support to others 


10.4 


6.5 


3.9 


0.304 


60.6 


0.13 




Financial worry 


2.5 


2.7 


-0.2 


0.307 


-6.0 


-0.13 




Residential well-being 
















Number of moves in past 3 years 


1.4 


1.6 


-0.2 


0.230 


-12.9 


-0.13 




Satisfaction with housing 


3.0 


3.2 


-0.2 


0.113 


-5.9 


-0.20 




Bad housing conditions 


1.1 


1.1 


0.0 


0.478 


-1.6 


-0.09 




Crowded housing 


1.5 


1.5 


0.0 


0.475 


-2.6 


-0.08 




Neighborhood good to raise children 


2.8 


3.0 


-0.2 


0.197 


-5.7 


-0.17 




Neighborhood resources 


3.5 


3.6 


0.0 


0.865 


-0.6 


-0.02 




Collective efficacy 


3.9 


4.0 


0.0 


0.877 


-0.6 


-0.02 




Housing safety: observer 


1.4 


1.4 


0.0 


0.282 


3.3 


0.15 




Neighborhood problems: observer 


8.8 


9.1 


-0.4 


0.393 


-4.2 


-0.12 




Physical well-being 
















Physical health 


3.5 


3.5 


0.0 


0.755 


1.3 


0.04 




Health condition hinders work (%) 


20.2 


21.7 


-1.5 


0.786 


-7.1 


-0.04 




Hospitalizations in last year 


0.1 


0.8 


-0.6 * 


0.058 


-81.6 


-0.36 




Substance use/abuse 


1.4 


1.3 


0.1 ♦♦ 


0.017 


10.5 


0.31 
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Appendix Table £.4.1 (continued) 













P- Value for 












Difference 




Program 


Control 




P- Value for 


% Effect Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact Size“ Barrier Groups^ 


Coping strategies and instrumental 












behaviors 












Sustainability 


3.0 


3.0 


0.1 


0.587 


1 .9 0.07 


Pursuing any goals (%) 


78.1 


82.0 


-3.9 


0.476 


-4.7 -0.10 


Achieving any goals (%) 


69.0 


71.6 


-2.6 


0.670 


-3.6 -0.06 


Awareness of helping resources 


2.0 


1.9 


0.1 


0.120 


4.8 0.21 


Community involvement 


1.4 


1.3 


0.0 


0.471 


2.0 0.09 


Married and living with spouse (%) 


25.4 


26.3 


-0.9 


0.867 


-3.5 -0.02 


Not married and living with partner (%) 


22.4 


26.1 


-3.7 


0.583 


-14.2 -0.08 


Discouragement of working 


2.4 


2.7 


-0.2 


0.126 


-9.1 -0.23 


Practical supports for worldng 


2.0 


2.0 


0.0 


0.562 


2.5 0.08 


Used EITC last year (%) 


74.3 


65.8 


8.5 


0.182 


12.9 0.18 


Aware of EITC last year (%) 


94.5 


88.2 


6.3 ♦ 


0.088 


7.2 0.23 


Psychosocial well-being 












General life stress 


2.5 


2.5 


0.0 


0.887 


-0.7 -0.02 


Time pressure 


3.3 


3.3 


0.0 


0.943 


0.3 0.01 


Job quality 


0.7 


0.7 


0.0 


0.896 


-1.1 -0.02 


Difficult life circumstances 


0.2 


0.2 


0.0 


0.444 


9.2 0.10 


Depression 


13.9 


15.4 


-1.5 


0.274 


-9.5 -0.14 


Religiosity 


3.7 


3.7 


0.0 


0.840 


-0.5 -0.03 


Hope 


3.0 


3.0 


0.1 


0.314 


2.5 0.13 


Influence of close others 


3.8 


3.5 


0.2 ♦♦ 


0.043 


6.0 0.28 


Availability of supportive others 


3.1 


3.1 


-0.1 


0.647 


-1.7 -0.06 


Perception of life now versus 












5 years ago 


4.3 


4.3 


0.0 


0.823 


0.7 0.03 


Sample size 


183 


181 








Two potential barriers or more 












Material well-being 












Family income: survey report ($) 


21,543 


18,001 


3,542 


0.145 


19.7 0.22 


Standard of living 


3.7 


3.8 


-0.1 


0.598 


-2.4 -0.09 


Material hardship 


0.2 


0.2 


0.1 


0.151 


30.8 0.24 


Food security 


3.3 


3.5 


-0.2 


0.216 


-4.7 -0.21 


Financial well-being 


15.4 


16.5 


-1.2 


0.124 


-7.2 -0.25 


Financial health 


0.4 


0.4 


0.0 


0.214 


-11.4 -0.20 


Financial support to others 


4.8 


13.6 


-8.8 * 


0.081 


-64.7 -0.30 


Financial worry 


2.7 


2.4 


0.3 


0.136 


11.9 0.24 


Residential well-being 












Number of moves in past 3 years 


1.6 


1.3 


0.3 , 


0.451 


20.2 0.17 


Satisfaction with housing 


3.0 


2.8 


0.1 


0.384 


5.3 0.16 


Bad housing conditions 


1.2 


1.1 


0.0 


0.709 


1.2 0.07 


Crowded housing 


1.5 


1.5 


0.0 


0.791 


1.0 0.03 
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Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size® 


P- Value for 
Difference 
Across 

Barrier Groups^ 


Neighborhood good to raise children 


2.9 


2.8 


0.1 


0.622 


2.9 


0.08 




Neighborhood resources 


3.5 


3.3 


0.2 


0.111 


7.3 


0.26 




Collective efficacy 


3.7 


3.7 


0.0 


0.993 


0.1 


0.00 




Housing safety: observer 


1.5 


1.4 


0.1 ♦ 


0.068 


7.1 


0.32 




Neighborhood problems: observer 


7.7 


8.2 


-0.6 


0.285 


-6.7 


-0.18 




Physical well-being 


Physical health 


3.5 


3.3 


0.2 


0.281 


6.3 


0.18 




Health condition hinders work (%) 


23.2 


19.3 


3.8 


0.585 


19.9 


0.09 




Hospitalizations in last year 


0.4 


0.2 


0.2 


0.316 


119.4 


0.13 




Substance use/abuse 


1.4 


1.4 


0.0 


0.561 


3.3 


0.11 




Coping strategies and instrumental 
behaviors 


Sustainability 


2.9 


3.0 


-0.1 


0.429 


-3.8 


-0.15 




Pursuing any goals (%) 


86.3 


85.4 


0.9 


0.879 


1.1 


0.02 




Achieving any goals (%) 


68.5 


72.0 


-3.5 


0.644 


-4.8 


-0.08 




Awareness of helping resources 


2.0 


1.9 


0.1 


0.123 


6.7 


0.29 




Community involvement 


1.4 


1.3 


0.1 * 


0.081 


5.9 


0.27 




Married and living with spouse (%) 


22.9 


22.1 


0.8 


0.898 


3.6 


0.02 




Not married and living with partner (%) 


26.7 


34.5 


-7.8 


0.398 


-22.7 


-0.18 




Discouragement of working 


2.5 


2.5 


0.0 


0.875 


-1.1 


-0.03 




Practical supports for working 


2.2 


2.1 


0.1 


0.619 


2.8 


0.09 




Used EITC last year (%) 


68.2 


71.3 


-3.1 


0.703 


-4.3 


-0.07 




Aware of EITC last year (%) 


93.0 


85.6 


7.4 


0.129 


8.6 


0.27 




Psychosocial well-being 


General life stress 


2.7 


2.4 


0.3 * 


0.064 


11.0 


0.30 




Time pressure 


3.4 


3.2 


0.2 


0.186 


7.6 


0.22 




Job quality 


0.5 


0.7 


-0.1 ♦ 


0.076 


-20.1 


-0.34 




Difficult life circumstances 


0.2 


0.2 


0.0 


0.551 


8.2 


0.11 




Depression 


14.8 


15.9 


-1.1 


0.541 


-7.0 


-0.10 




Religiosity 


3.7 


3.6 


0.2 


0.174 


4.3 


0.21 




Hope 


2.9 


3.1 


-0.2 


0.136 


-4.9 


-0.28 




Influence of close others 


3.6 


3.7 


-0.1 


0.331 


-3.5 


-0.17 




Availability of supportive others 


3.0 


3.0 


-0.1 


0.679 


-1.9 


-0.07 




Perception of life now versus 


5 years ago 


4.2 


4.3 


-0.1 


0.651 


-1.9 


-0.08 




Sample size 


125 


148 
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Appendix Table E.4.1 (continued) 

SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent 
Table 4. 1 presents the range of outcomes reported in this table. 

Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both grovaps combined. This standard deviation is always obtained from the entire 
research sample, even if the table shows impacts for subgroups. 

'’a statistical test was conducted to measure whether impacts differed significantly across the subgroiq) dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt ~ 1 percent, ft = 5 percent, and t 
= 10 percent. 



The New Hope Project 
Appendix Table E.4.2 

Impacts on Parental Well-Being, by Ethnicity 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size" 


P-Value for 
Difference 
Across 
Ethnicities^ 


African-American 
Material well-being 


Family income: survey report ($) 


19,928 


19,661 


266 


0.869 


1.4 


0.02 


0.436 


Standard of living 


3.7 


3.6 


0.2 


0.173 


4.9 


0.17 


0.414 


Material hardship 


0.2 


0.2 


0.0 


0.804 


-3.4 


-0.03 


0.268 


Food security 


3.4 


3.4 


0.0 


0.726 


-1.0 


-0.04 


0.139 


Financial well-being 


16.2 


16.2 


-0.1 


0.890 


-0.5 


-0.02 


0.516 


Financial health 


0.5 


0.5 


0.0 


0.502 


-3.9 


-0.07 


0.660 


Financial support to others 


10.3 


12.4 


-2.0 


0.592 


-16.5 


-0.07 


0.904 


Financial worry 


2.4 


2.4 


0.1 


0.645 


2.5 


0.05 


0.418 


Residential well-being 


Number of moves in past 3 years 


1.1 


1.0 


0.1 


0.674 


8.6 


0.05 


0.999 


Satisfaction with housing 


2.9 


2.9 


0.0 


0.995 


0.0 


0.00 


0.384 


Bad housing conditions 


1.1 


1.1 


0.0 


0.172 


-2.6 


-0.16 


0.068 t 


Crowded housing 


1.5 


1.6 


0.0 


0.671 


-1.7 


-0.05 


0.850 


Neighborhood good to raise children 


3.0 


2.9 


0.1 


0.444 


3.0 


0.09 


0.819 


Neighborhood resources 


3.5 


3.3 


0.2 * 


0.091 


5.7 


0.20 


0.600 


Collective efficacy 


3.8 


3.8 


0.0 


0.865 


-0.6 


-0.02 


0.613 


Housing safety: observer 


1.5 


1.4 


0.0 


0.554 


1.5 


0.07 


0.960 


Neighborhood problems: observer 


8.4 


8.0 


0.4 


0.212 


5.4 


0.14 


0.293 


Physical well-being 


Physical health 


3.6 


3.4 


0.2 


0.160 


5.4 


0.16 


0.070 t 


Health condition hinders work (%) 


20.9 


19.8 


1.0 


0.829 


5.3 


0.03 


0.958 


Hospitalizations in last year 


0.1 


0.5 


-0.4* 


0.086 


-77.2 


-0.22 


0.227 


Substance use/abuse 


1.3 


1.3 


0.0 


0.456 


2.9 


0.09 


0.575 


Coping strategies and instrumental 
behaviors 


Sustainability 


3.0 


2.9 


0.1 


0.248 


3.7 


0.14 


0.531 


Pursuing any goals (%) 


84.1 


85.2 


-1.0 


0.805 


-1.2 


-0.03 


0.005 ttt 


Achieving any goals (%) 


73.2 


70.4 


2.8 


0.593 


4.0 


0.06 


0.875 


Awareness of helping resources 


2.0 


1.9 


0.1 


0.120 


4.1 


0.18 


0.428 


Community involvement 


1.4 


1.4 


0.0 


0.542 


1.4 


0.07 


0.929 


Married and living with spouse (%) 


16.9 


15.0 


1.9 


0.644 


12.4 


0.04 


0.676 


Not married and living with partner (%) 


22.0 


25.0 


-3.0 


0.583 


-12.0 


-0.07 


0.468 


Discouragement of working 


2.5 


2.5 


0.1 


0.639 


2.5 


0.06 


0.780 


Practical supports for working 


2.1 


2.1 


0.0 


0.928 


-0.3 


-0.01 


0.513 


Used EITC last year (%) 


71.2 


69.6 


1.6 


0.769 


2.3 


0.03 


0.462 


Aware of EITC last year (%) 


97.5 


92.3 


5.2 ♦♦ 


0.041 


5.6 


0.19 


0.986 
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Appendix Table E.4.2 (continued) 



P-Value for 
Difference 

Program Control P -Value for % Effect Across 



Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size^ 


Ethnicities 


Psychosocial well-being 


General life stress 


2.5 


2.5 


0.0 


0.650 


-1.8 


-0.05 


0.493 


Time pressure 


3.4 


3.3 


0.1 


0.551 


2.3 


0.07 


0.003 ttt 


Job quality 


0.6 


0.7 


-0.1 


0.161 


-9.7 


-0.17 


0.127 


Difficult life circumstances 


0.2 


0.2 


0.0 


0.452 


-6.3 


-0.09 


0.025 tt 


Depression 


14.8 


16.4 


-1.6 


0.199 


-9.7 


-0.15 


0.231 


Religiosity 


3.8 


3.8 


0.1 


0.213 


2.4 


0.13 


0.932 


Hope 


3.0 


3.0 


0.0 


0.742 


-0.7 


-0.04 


0.121 


Influence of close others 


3.7 


3.6 


0.1 


0.382 


2.2 


0.10 


0.708 


Availability of supportive others 


3.1 


3.0 


0.0 


0.652 


1.5 


0.05 


0.847 


Perception of life now versus 


5 years ago 


4.3 


4.2 


0.0 


0.841 


0.6 


0.02 


0.894 


Sample size 


253 


228 












Hispanic 

Material well-being 


Family income: survey report ($) 


27,878 


22,988 


4,889 


0.135 


21.3 


0.30 




Standard of living 


3.9 


4.0 


-0.1 


0.568 


-2.2 


-0.09 




Material hardship 


0.2 


0.1 


0.0 


0.687 


10.4 


0.06 




Food security 


3.5 


3.5 


-0.1 


0.486 


-2.2 


-0.10 




Financial well-being 


16.5 


17.2 


-0.7 


0.361 


-4.0 


-0.14 




Financial health 


0.5 


0.5 


0.0 


0.542 


4.6 


0.09 




Financial support to others 


6.0 


7.0 


-1.0 


0.823 


-13.7 


-0.03 




Financial worry 


2.9 


3.0 


-0.1 


0.666 


-3.1 


-0.07 




Residential well-being 


Number of moves in past 3 years 


1.4 


1.3 


0.1 


0.504 


10.3 


0.09 




Satisfaction with housing 


3.0 


3.2 


-0.3 ♦ 


0.098 


-7.8 


-0.28 




Bad housing conditions 


1.1 


1.1 


0.1 ♦ 


0.084 


5.1 


0.29 




Crowded housing 


1.6 


1.6 


0.0 


0.705 


1.8 


0.06 




Neighborhood good to raise children 


3.4 


3.6 


-0.2 


0.176 


-6.1 


-0.22 




Neighborhood resources 


3.6 


3.6 


0.0 


0.995 


0.0 


0.00 




Collective efficacy 


4.0 


3.8 


0.2 


0.388 


4.7 


0.15 




Housing safety: observer 


1.4 


1.4 


0.0 


0.495 


2.8 


0.12 




Neighborhood problems: observer 


9.0 


9.6 


-0.5 


0.357 


-5.6 


-0.17 




Physical well-being 


Physical health 


.3.5 


3.6 


-0.2 


0.365 


-4.7 


-0.15 




Health condition hinders work (%) 


19.3 


16.7 


2.6 


0.683 


15.5 


0.06 




Hospitalizations in last year 


0.5 


0.4 


0.0 


0.965 


3.2 


0.01 




Substance use/abuse 


1.3 


1.2 


0.1 


0.222 


6.9 


0.20 
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P- Value for 
Difference 





Program 


Control 




P- Value for 


% 


Effect Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size* Ethnicities^ 


Coping strategies and instrumental 














behaviors 














Sustainability 


3.1 


3.1 


0.0 


0.828 


-0.8 


-0.03 


Pursuing any goals (%) 


70.6 


91.7 


-21.1 ♦♦♦ 


0.002 


-23.0 


-0.55 


Achieving any goals (%) 


69.5 


67.2 


2.3 


0.769 


3.4 


0.05 


Awareness of helping resources 


2.0 


1.8 


0.2 ♦♦ 


0.012 


10.3 


0.43 


Community involvement 


1.3 


1.3 


0.0 


0.646 


1.6 


0.08 


Married and living with spouse (%) 


32.6 


27.0 


5.6 


0.441 


20.7 


0.14 


Not married and living with partner (%) 


39.3 


34.7 


4.7 


0.637 


13.4 


0.10 


Discouragement of working 


2.5 


2.6 


-0.1 


0.688 


-3.0 


-0.07 


Practical supports for working 


2.0 


2.0 


0.0 


0.892 


0.8 


0.02 


Used EITC last year (%) 


66.2 


63.5 


2.7 


0.753 


4.3 


0.06 


Aware of EITC last year (%) 


85.6 


81.4 


4.2 


0.476 


5.1 


0.15 


Psychosocial well-being 














General life stress 


2.5 


2.3 


0.2 


0.314 


6.6 


0.17 


Time pressure 


3.2 


3.2 


0.0 


0.831 


1.2 


0.04 


Job quality 


0.7 


0.7 


0.1 


0.298 


11.4 


0.19 


Difficult life circumstances 


0.2 


0.1 


0.1 ♦♦♦ 


0.009 


53.8 


0.40 


Depression 


15.4 


14.4 


1.0 


0.595 


7.0 


0.09 


Religiosity 


3.8 


3.6 


0.1 


0.231 


3.7 


0.18 


Hope 


3.0 


3.2 


-0.2 ♦♦ 


0.031 


-6.3 


-0.37 


Influence of close others 


3.7 


3.7 


0.0 


0.773 


-1.0 


-0.05 


Availability of supportive others 


3.1 


3.1 


0.0 


0.894 


-0.6 


-0.02 


Perception of life now versus 














5 years ago 


4.4 


4.4 


0.0 


0.833 


-0.8 


-0.03 


Sample size 


125 


126 










White 














Material well-being 














Family income: survey report ($) 


24,762 


24,759 


3 


1.000 


0.0 


0.00 


Standard of living 


4.1 


3.9 


0.1 


0.594 


3.4 


0.13 


Material hardship 


0.1 


0.2 


-0.1 


0.111 


-43.6 


-0.45 


Food security 


3.6 


3.3 


0.3 * 


0.073 


10.3 


0.45 


Financial well-being 


16.3 


15.2 


1.1 


0.436 


7.2 


0.23 


Financial health 


0.5 


0.5 


0.0 


0.874 


2.6 


0.05 


Financial support to others 


7.7 


12.8 


-5.1 


0.537 


-39.9 


-0.17 


Financial worry 


2.2 


2.5 


-0.4 


0.241 


-14.2 


-0.29 


Residential well-being 














Number of moves in past 3 years 


1.4 


1.3 


0.1 


0.779 


10.2 


0.08 


Satisfaction with housing 


3.2 


3.2 


0.0 


0.877 


-0.8 


-0.03 


Bad housing conditions 


1.1 


1.1 


0.0 


0.424 


-3.5 


-0.21 


Crowded housing 


1.5 


1.5 


0.0 


0.953 


0.5 


0.02 
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Appendix Table £.4.2 (continued) 



P-Value for 
Difference 





Program 


Control 




P-Value for 


% 


Effect Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size^ Ethnicities^ 


Neighborhood good to raise children 


3.7 


3.4 


0.3 


0.350 


8.1 


0.27 


Neighborhood resources 


3.9 


3.8 


0.1 


0.709 


2.2 


0.09 


Collective efficacy 


4.1 


4.2 


-0.2 


0.634 


-3.8 


-0.14 


Housing safety: observer 


1.3 


1.3 


0.0 


0.851 


1.1 


0.05 


Neighborhood problems: observer 


9.4 


9.9 


-0.5 


0.640 


-4.6 


-0.15 


Physical well-being 














Physical health 


3.4 


2.9 


0.5 ** 


0.039 


18.8 


0.48 


Health condition hinders work (%) 


32.2 


27.6 


4.7 


0.729 


16.9 


0.11 


Hospitalizations in last year 


0.3 


0.2 


0.1 


0.522 


80.6 


0.08 


Substance use/abuse 


1.6 


1.6 


0.0 


0.656 


-2.9 


-0.11 


Coping strategies and instrumental 














behaviors 














Sustainability 


3.0 


2.8 


0.2 


0.327 


7.5 


0.27 


Pursuing any goals (%) 


81.1 


60.6 


20.5 


0.127 


33.8 


0.53 


Achieving any goals (%) 


70.1 


60.7 


9.5 


0.453 


15.6 


0.21 


Awareness of helping resources 


1.9 


1.9 


0.0 


0.739 


2.2 


0.10 


Community involvement 


1.3 


1.3 


0.0 


0.877 


-1.0 


-0.05 


Married and living with spouse (%) 


27.6 


34.2 


-6.6 


0.577 


-19.2 


-0.16 


Not married and living with partner (%) 


33.0 


16.4 


16.6 


0.326 


100.8 


0.37 


Discouragement of working 


2.7 


2.8 


-0.1 


0.734 


-3.7 


-0.10 


Practical supports for working 


2.2 


2.0 


0.2 


0.226 


10.1 


0.31 


Used EITC last year (%) 


78.1 


59.5 


18.6 


0.149 


31.2 


0.40 


Aware of EITC last year (%) 


99.9 


94.6 


5.3 


0.255 


5.7 


0.19 


Psychosocial well-being 














General life stress 


2.5 


2.6 


-0.1 


0.667 


-3.6 


-0.11 


Time pressure 


3.0 


3.8 


-0.8 *** 


0.001 


-21.3 


-0.75 


Job quality 


0.5 


0.7 


-0.2 


0.159 


-24.8 


-0.45 


Difficult life circumstances 


0.2 


0.2 


0.0 


0.562 


-12.3 


-0.16 


Depression 


11.5 


16.2 


-4.8 


0.112 


-29.3 


-0.44 


Religiosity 


3.3 


3.1 


0.1 


0.520 


4.8 


0.21 


Hope 


2.9 


2.9 


0.1 


0.472 


2.9 


0.15 


Influence of close others 


3.8 


3.7 


0.1 


0.540 


3.4 


0.16 


Availability of supportive others 


3.2 


3.3 


-0.1 


0.721 


-2.3 


-0.09 


Perception of life now versus 














5 years ago 


4.2 


4.3 


-0.1 


0.699 


-2.2 


-0.10 


Sample size 


44 


66 
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Appendix Table E.4.2 (continued) 



SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 4. 1 presents the range of outcomes reported in this table. 

Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and t = 
10 percent. 
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Parenting behavior 

Monitoring 3.2 3.2 0.0 0.975 -0.1 0.00 0.744 

Prevention-of-harm strategies 3^9 3^9 0^0 0.740 -0.8 -0.04 0.100 

(continued) 
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Parenting stress 1.8 1.8 0.0 0.712 -2.4 -0.06 

Prevention-of-harm confidence 3.7 3.6 0.1 0.621 2.3 0.09 

Positive youth-parent relations 4.2 4.2 0.0 0.905 0.3 0.02 

Child-reported positive relations 4A 4J M 0.569 -0.9 -0.07 
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The New Hope Project 
Appendix Table E.5.2 



Impacts on Parenting and Parent-Child Relations, by Ethnicity 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P -Value for 
Difference 


% 

Impact 


Effect 

Size" 


P-Value for 
Difference 
Across 
Ethnicities^ 


African-American 
















Effective child management 


3.9 


3.8 


0.1 ♦♦ 


0.023 


3.8 


0.24 


0.059 t 


Problems with control 


2.2 


2.5 


-0.2 ♦♦ 


0.019 


-10.1 


-0.26 


0.114 


Frequency of discipline 


2.0 


2.1 


-0.1 


0.146 


-4.5 


-0.16 


0.306 


Parenting stress 


1.8 


1.9 


-0.1 


0.154 


-5.9 


-0.15 


0.406 


Prevention-of-harm confidence 


3.7 


3.6 


0.2 


0.107 


4.2 


0.17 


0.203 


Positive youth-parent relations 


4.2 


4.2 


0.1 


0.411 


1.5 


0.10 


0.446 


Child-reported positive relations 


4.4 


4.5 


0.0 


0.693 


-0.5 


-0.04 


0.741 


Child-reported acceptance and involvement 


3.4 


3.4 


0.0 


0.400 


1.4 


0.10 


0.591 


Child-reported monitoring 


3.2 


3.1 


0.1 


0.115 


4.2 


0.19 


0.120 


Negative youth-parent relations 


2.8 


2.7 


0.1 


0.413 


2.4 


0.10 


0.770 


Child-reported negative relations 


2.6 


2.5 


0.1 


0.327 


3.2 


0.10 


0.536 


Child-reported autonomy 


2.4 


2.5 


-0.1 


0.283 


-2.9 


-0.13 


0.822 


Warm and structured parenting 


3.9 


3.8 


0.1 


0.359 


1.8 


0.11 


0.954 


Parent-reported warmth 


4.7 


4.6 


0.1 


0.531 


1.8 


0.07 


0.759 


Observer-reported warmth 


2.4 


2.4 


0.0 


0.742 


1.1 


0.04 


0.203 


Regularity of family routines 


3.8 


3.7 


0.1 


0.235 


3.0 


0.15 


0.051 t 


Parenting behavior 
















Monitoring 


3.2 


3.1 


0.0 


0.454 


1.5 


0.08 


0.014tt 


Prevention-of-harm strategies 


4.0 


3.8 


0.1 


0.042 


3.8 


0.19 


0.036 tt 


Hispanic 
















Effective child management 


4.0 


4.1 


-0.1 


0.374 


-2.0 


-0.13 




Problems with control 


2.1 


2.0 


0.1 


0.602 


3.9 


0.08 




Frequency of discipline 


1.8 


1.7 


0.1 


0.567 


2.9 


0.08 




Parenting stress 


1.7 


1.6 


0.0 


0.708 


2.4 


0.05 




Prevention-of-harm confidence 


3.7 


3.9 


-0.1 


0.455 


-3.2 


-0.14 




Positive youth-parent relations 


4.2 


4.3 


0.0 


0.697 


-0.9 


-0.07 




Child-reported positive relations 


4.3 


4.4 


-0.1 


0.411 


-1.8 


-0.13 




Child-reported acceptance and involvement 


3.4 


3.4 


-0.1 


0.492 


-1.5 


-0.11 




Child-reported monitoring 


3.2 


3.3 


-0.1 


0.624 


-1.8 


-0.09 




Negative youth-parent relations 


2.8 


2.8 


0.0 


0.921 


-0.4 


-0.02 




Child-reported negative relations 


2.6 


2.5 


0.2 


0.248 


6.4 


0.19 




Child-reported autonomy 


2.4 


2.4 


0.0 


0.999 


0.0 


0.00 




Warm and structured parenting 


4.0 


3.9 


0.0 


0.655 


1.2 


0.07 




Parent-reported warmth 


4.8 


4.6 


0.2 


0.374 


3.8 


0.15 




Observer-reported warmth 


2.4 


2.3 


0.1 


0.550 


2.7 


0.09 




Regularity of family routines 


3.9 


4.0 


-0.1 


0.228 


-3.4 


-0.18 
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Appendix Table E.5.2 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% Effect 
Impact Size“ 


P-Value for 
Difference 
Across 
Ethnicities^ 


Parenting behavior 


Monitoring 


3.2 


3.4 


-0.3 ♦♦ 


0.030 


-7.4 -0.44 




Prevention -of-harm strategies 


3.7 


3.9 


-0.1 


0.366 


-3.5 -0.18 




White 


Effective child management 


3.9 


3.8 


0.1 


0.467 


2.9 0.18 




Problems with control 


2.3 


2.4 


-0.1 


0.575 


-4.7 -0.11 




Frequency of discipline 


1.9 


1.9 


0.0 


0.952 


-0.4 -0.01 




Parenting stress 


1.8 


2.0 


-0.1 


0.360 


-7.2 -0.19 




Prevention-of-harm confidence 


3.8 


3.6 


0.2 


0.516 


4.3 0.18 




Positive youth-parent relations 


4.0 


4.2 


-0.2 


0.362 


-5.1 -0.37 




Child-reported positive relations 


4.4 


4.3 


0.0 


0.757 


0.9 0.06 




Child-reported acceptance and involvement 


3.4 


3.3 


0.1 


0.584 


2.5 0.18 




Child-reported monitoring 


2.9 


3.3 


-0.4 


0.116 


-11.0 -0.53 




Negative youth-parent relations 


2.9 


2.7 


0.2 


0.575 


5.9 0.26 




Child-reported negative relations 


2.6 


2.7 


-0.1 


0.613 


-3.7 -0.12 




Child-reported autonomy 


2.4 


2.5 


0.0 


0.799 


-1.9 -0.09 




Warm and structured parenting 


3.9 


3.9 


0.0 


0.842 


0.8 0.05 




Parent-reported warmth 


4.7 


4.4 


0.2 


0.293 


5.6 0.22 




Observer-reported warmth 


2.3 


2.6 


-0.3 


0.200 


-10.5 -0.41 




Regularity of family routines 


3.9 


3.7 


0.2 


0.141 


6.8 0.33 




Parenting behavior 


Monitoring 


3.3 


3.3 


0.0 


0.787 


1.1 0.06 




Prevention-of-harm strategies 


3.6 


3.8 


-0.2 


0.322 


-5.4 -0.27 





SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Table 5.1 presents the ranges of outcomes reported in this table. 

For Afiican -Americans, parent reports were available for 462 children; and child reports were available for 462 
children. For Hispanics, parent reports were available for 236 children; and child reports were available for 236 children. 
For whites, parent reports were available for 109 children; and child reports were available for 109 children. Actual sample 
sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is^al ways obtained from the entire research 
sample, even if the table shows impacts for subgroiq)s. 

^A statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant Statistical significance levels are indicated as ttt = i percent, tt = 5 percent, and | = 
10 percent. 
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Potential Barriers to Employment 



o 

O 

A 

B 

s 

Z, 

A 



o 



U 



J 3 

u 

c 

o 



u 

A 

Q. 

E 



s g s o 

CQ 

o 

u 



c 

o 

u 



d 

o 



cd 

d 



§ I 

rL Q 



d 

o 

Urn 

o 



E gJ 
2 o 
^ rfe 
P o 



CN 

»T) 

O 

o 



o 



o 

o 



3 

B 

C 

3 

V 

"S 

o 



.2 .2 

a c 

A & 

UD ^ 

s £ 

T B 

=* 2 
Q S 



>% 

c 

< 



(N VO 0\ 
OO CTv 
oo ^ Tj- 

o o o 



^ oo m 

CM ^ ^ 

o o' o' 



rn (N VO 
^ ov 

(N (N ^ 



m (N 
(N 00 VO 
(N (N ^ 

c 5 c 5 o' 



o 

ri 



E 

o 

<U 

E .«> 

o ^ 



E 



O 4= cd 
4 = " 

C “ 

< 



VO o 

m VO ^ ^ 
(N 

o o' o' o 



(N C^ VO OO 
(N O ^ ^ 

o' o o' o 



00 VO o 
00 VO uS rn 
(N (N (N 

I I I 1 



00 ^ O vn 
oo ^ vn 
^ ^ (N 

o c 5 o' o 



OV r-; 00 (N 

o o' o' o' o' o' o 



tN rn 
^ tN ri 



00 ov 

rsi o' ^ ^ 



rn ov ov O rn 

rn ^ ^ ri o' ^ o' 



TJ 
u 

trt 

•? « > 
52 =o g> 



cd 

o 

. Lm 

*0 o 
(U c 
t /5 .£ 

t ^ 

D Cd 

S' >' 

o 



.fi 

s 

3 

B 

C 

O 

s 

s I 

00 ^ 






P O 

Jd ^ 

O T 3 
P 



VO 

»T) 

vq 

o 



.2 .5 

a. E 
^ a. 

^ ^ f/i 

OX) 52 
E 



££ g>u&ou So 



Q S 



C^ (N Tf 

m (N (N 
00 r- 00 

o o' o 



ov ov r- 
^ (N O 

o o o 



^ ON r-; 
od Tf o 



Tf Tf »T) 

oo o o 
(N — ^ r- 



2 

cd 



>v 

c 

< 



cd ^ 

^ t; 

E ^ 
o ^ 
4= o 
c 

c ^ 

< 



E 

o 

O 4 = 

E J/3 

o h 



o 



VO 

(N 

o 



o 



00 

as 

o 



rn ^ 
o o o 



O m fNj 00 00 

O ^ ^ o o 



o rn 00 r- 

^ ^ o o 



*0 

a 

C/J 

'i 

o 

d 

3 

CO 

C 

3 

C 

< 



- 229 - 



er|c 



26 z 



o 



Care by a minor 0.2 0.2 -O.l 0.667 -24.7 -0.07 O.lll 

Self-care 0.3 0.5 -0.2 0.270 -34.4 -0.20 0.648 

Cared for sibling(s) 02 0.4 -0.2 0.106 -58.9 -0.26 0.389 
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Any home-based care: 13 17 .0.4 0.128 -20.9 -0.24 

In child's home 10 I.3 _o.3 0.261 -21.0 -0.19 

In caregiver’s home OJ 0.7 0.0 0.995 -0.2 0.00 
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The New Hope Project 
Appendix Table E.5.4 



Impacts on Child Care Use and Monthly Child Care Costs, by Ethnicity 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size" 


P-Value for 
Difference 
Across 
Ethnicities^ 


African-American 

During the prior school year, number 
of months spent in 


Any formal care 


3.0 


2.5 


0.5 


0.269 


20.4 


0.13 


0.729 


Any home-based care: 


4.0 


4.3 


-0.3 


0.541 


-6.9 


-0.07 


0.035 ft 


In child’s home 


2.5 


3.2 


-0.6 


0.191 


-19.8 


-0.15 


0.017 ft 


In caregiver’s home 


2.3 


2.1 


0.1 


0.745 


6.7 


0.04 


0.323 


Any unsupervised care: 


2.2 


2.7 


-0.5 


0.282 


-17.9 


-0.13 


0.101 


Care by a minor 


0.9 


1.1 


-0.2 


0.519 


-18.3 


-0.08 


0.200 


Self-care 


0.9 


1.6 


-0.6 * 


0.057 


-41.9 


-0.24 


0.071 t 


Cared for sibling(s) 


1.0 


1.2 


-0.2 


0.594 


-16.0 


-0.07 


0.009 ttt 


During the prior summer, 
number of months spent in 


Any formal care 


0.9 


0.6 


0.3 ♦♦ 


0.047 


49.5 


0.24 


0.995 


Any home-based care: 


1.4 


1.6 


-0.2 


0.327 


-10.3 


-0.11 


0.294 


In child’s home 


1.0 


1.1 


-0.2 


0.355 


-13.9 


-0.11 


0.026 tt 


In caregiver’s home 


0.8 


0.8 


0.0 


0.860 


-3.3 


-0.02 


0.553 


Any unsupervised care: 


0.6 


0.8 


-0.2 


0.174 


-24.5 


-0.17 


0.124 


Care by a minor 


0.2 


0.3 


-0.1 


0.156 


-45.4 


-0.16 


0.187 


Self-care 


0.3 


0.4 


-0.1 


0.298 


-26.4 


-0.12 


0.383 


Cared for sibling(s) 


0.3 


0.3 


0.0 


0.824 


-7.7 


-0.03 


0.024 tt 


During the prior year, 
number of months spent in 


Any formal care 


3.5 


2.8 


0.7 


0.167 


25.7 


0.16 


0.614 


Any home-based care 


5.3 


5.8 


-0.5 


0.376 


-8.9 


-0.10 


0.043 ft 


Any unsupervised care 


2.8 


3.5 


-0.7 


0.200 


-20.0 


-0.15 


0.078 t 


Out-of-pocket child care costs in 


41.0 


45.9 


-4.9 


0.719 


-10.7 


-0.04 


0.234 


prior month ($) 


Sample size 


246 


217 













(continued) 
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Appendix Table £.5.4 (continued) 



P-Value for 
Difference 

Program Control P- Value for % Effect Across 

Outcome Group Group Difference Difference Impact Size^ Ethnicities^ 

Hispanic 



During the prior school year, 
number of months spent in 



Any formal care 


2.6 


1.3 


1.3 ♦ 


0.057 


97.4 


0.33 


Any home-based care: 


3.5 


4.6 


-1.1 


0.169 


-23.5 


-0.26 


In child’s home 


2.5 


3.4 


-0.9 


0.276 


-25.2 


-0.21 


In caregiver's home 


1.9 


2.3 


-0.4 


0.572 


-16.8 


-0.11 


Any unsupervised care: 


2.0 


1.8 


0.2 


0.760 


9.5 


0.05 


Care by a minor 


1.0 


1.1 


-0.1 


0.882 


-6.1 


-0.02 


Self-care 


0.7 


0.4 


0.2 


0.489 


58.9 


0.09 


Cared for sibling(s) 


1.0 


0.7 


0.2 


0.532 


33.1 


0.09 


During the prior summer. 














number of months spent in 














Any formal care 


1.0 


0.7 


0.3 


0.241 


36.5 


0.20 


Any home-based care: 


1.3 


1.6 


-0.4 


0.160 


-22.1 


-0.25 


In child’s home 


0.9 


1.3 


-0.4 


0.117 


-31.2 


-0.28 


In caregiver’s home 


0.6 


0.6 


0.0 


0.802 


8.0 


0.04 


Any unsupervised care: 


0.5 


0.6 


-0.1 


0.460 


-21.4 


-0.12 


Care by a minor 


0.2 


0.4 


-0.2 


0.211 


-44.3 


-0.23 


Self-care 


0.3 


0.2 


0.1 


0.672 


29.0 


0.07 


Cared for sibling(s) 


0.3 


0.2 


0.1 


0.669 


29.5 


0.07 


During the prior year. 














number of months spent in 














Any formal care 


4.0 


2.6 


1.5 ♦ 


0.060 


58.3 


0.32 


Any home-based care 


4.8 


6.1 


-1.3 


0.169 


-21.6 


-0.25 


Any unsupervised care 


2.4 


2.4 


0.1 


0.930 


2.6 


0.01 


Out-of-pocket child care costs in 


60.1 


31.2 


28.8 


0.169 


92.4 


0.25 


prior month ($) 














Sample size 


119 


123 
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Appendix Table E.5.4 (continued) 



Outcome 


Program 

Group 


Control 

Group 


P-Value for 
Difference Difference 


% 

Impact 


P-Value for 
Difference 
Effect Across 

Size^ Ethnicities^ 


White 














During the prior school year, 














number of months spent in 














Any formal care 


2.3 


1.6 


0.7 


0.493 


44.8 


0.19 


Any home ‘based care: 


2.8 


5.8 


-3.0 ♦♦ 


0.016 


-50.9 


-0.69 


In child's home 


1.9 


5.5 


-3.6 ♦♦♦ 


0.004 


-65.4 


-0.90 


In caregiver's home 


1.4 


2.5 


-1.1 


0.315 


-43.6 


-0.30 


Any un supervised care: 


1.2 


3.0 


-1.8 ♦♦ 


0.018 


-60.9 


-0.49 


Care by a minor 


0.1 


1.4 


-1.2 ♦♦ 


0.020 


-90.4 


-0.46 


Self‘Care 


1.0 


1.6 


-0.6 


0.409 


-35.3 


-0.21 


Cared for sibling(s) 


0.1 


2.0 


-1.9 *** 


0.002 


-96.0 


-0.69 


During the prior summer, 














number of months spent in 














Any formal care 


1.0 


0.6 


0.4 


0.245 


68.8 


0.33 


Any home-based care: 


1.1 


1.8 


-0.7 * 


0.085 


-38.1 


-0.47 


In child's home 


0.8 


1.9 


-1.1 *** 


0.005 


-57.0 


-0.75 


In caregiver's home 


0.3 


0.6 


-0.3 


0.363 


-44.9 


-0.23 


Any unsupervised care: 


0.3 


1.0 


-0.7 *** 


0.003 


-73.5 


-0.60 


Care by a minor 


0.0 


0.5 


-0.5 *** 


0.006 


-99.8 


-0.58 


Self‘Care 


0.3 


0.5 


-0.2 


0.326 


-44.5 


-0.23 


Cared for sibling(s) 


0.0 


0.4 


-0.5 *** 


0.006 


-103.0 


-0.54 


During the prior year, 














number of months spent in 














Any formal care 


3.1 


2.1 


1.0 


0.383 


50.9 


0.23 


Any home-based care 


3.8 


7.6 


-3.8 ** 


0.015 


-49.5 


-0.72 


Any unsupervised care 


1.4 


4.0 


-2.6 *** 


0.007 


-64.5 


-0.55 


Out-of-pocket child care costs in 


20.1 


17.6 


2.5 


0.908 


14.2 


0.02 


prior month ($) 














Sample size 


42 


63 
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Appendix Table £.5.4 (continued) 

SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p- value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, ft “ 5 percent, and t 
== 1 0 percent. 





The New Hope Project 
Appendix Table £.5.5 

Impacts on Children's Activities, by Number of Parent's Potential Barriers to Employment 

P- Value for 
Difference 

Program Control P-Value for % Effect Across 



Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size® 


Barrier Groups 


No potential barriers 
Structured activities • school year 


Total structured activities: parent report 


2.4 


2.4 


0.0 


0.849 


1.1 


0.03 


0.584 


Lessons 


2.1 


2.1 


-0.1 


0.795 


-2.5 


-0.04 


0.631 


Sports with a coach 


2.3 


2.3 


0.0 


0.854 


1.7 


0.03 


0.366 


Club/youth group 


2.3 


2.3 


0.1 


0.760 


3.0 


0.05 


0.567 


Religious class or activity 


2.9 


2.8 


0.1 


0.795 


1.9 


0.04 


0.470 


Community center 


2.4 


2.3 


0.1 


0.780 


2.7 


0.04 


0.831 


School-related programs: parent report 


Program to help with school 


2.3 


2.2 


0.1 


0.762 


3.4 


0.05 


0.023 tt 


Before/after-school program 


2.2 


1.9 


0.3 


0.266 


15.1 


0.17 


0.721 


Total structured activities: child report 


2.6 


2.4 


0.2 


0.199 


8.7 


0.25 


0.466 


Lessons 


2.6 


2.4 


0.2 


0.437 


8.7 


0.13 


0.315 


Sports with a coach 


2.7 


2.6 


0.1 


0.774 


3.5 


0.06 


0.604 


Club/youth group 


2.3 


2.2 


0.1 


0.830 


2.9 


0.04 


0.761 


Religious class or activity 


3.2 


2.7 


0.5 


0.037 


20.6 


0.39 


0.181 


Community center 


2.5 


2.3 


0.1 


0.580 


6.4 


0.09 


0.892 


School-related programs: child report 


Program to help with school 


1.8 


2.1 


-0.4 


0.190 


-16.4 


-0.23 


0.320 


Before/after-school program 


1.9 


1.4 


0.5 


0.049 


36.8 


0.41 


0.262 


Structured activities - summer 


Total structured activities: parent report 


2.1 


2.2 


0.0 


0.776 


-2.1 


-0.05 


0.373 


Lessons 


1.6 


1.8 


-0.2 


0.293 


-11.5 


-0.17 


0.430 


Sports with a coach 


1.8 


2.0 


-0.2 


0.487 


-7.9 


-0.11 


0.437 


Club/youth group 


2.0 


2.1 


-0.1 


0.735 


-3.5 


-0.05 


0.052 t 


Religious class or activity 


2.8 


2.7 


0.2 


0.489 


5.7 


0.11 


0.648 


Community center 


2.3 


2.3 


0.0 


0.902 


1.3 


0.02 


0.813 


School-related programs: parent report 


Summer school 


2.2 


2.1 


0.1 


0.800 


3.6 


0.05 


0.306 


Daycamp 


1.9 


1.7 


0.2 


0.346 


13.3 


0.15 


0.113 


Total structured activities: child report 


2.1 


2.0 


0.1 


0.488 


5.2 


0.12 


0.918 


Lessons 


1.5 


1.6 


-0.1 


0.509 


-8.0 


-0.10 


0.868 


Sports with a coach 


1.9 


2.1 


-0.1 


0.609 


-6.3 


-0.08 


0.632 


Club/youth group 


1.9 


1.5 


0.4 ♦ 


0.098 


25.1 


0.27 


0.750 


Religious class or activity 


3.0 


2.4 


0.6 ♦♦ 


0.018 


26.2 


0.44 


0.199 


Community center 


2.0 


2.2 


-0.3 


0.404 


-11.5 


-0.16 


0.832 


School-related programs: child report 


Summer school 


1.7 


2.1 


-0.4 


0.227 


-18.2 


-0.23 


0.204 


Daycamp 


1.7 


1.6 


0.1 


0.791 


4.9 


0.05 


0.790 
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Appendix Table E.S.S (continued) 

















P-Value for 
Difference 




Program 


Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size" 


Barrier Groups^ 


Service and work for pay - school year 
















Service and volunteer: parent report 


1.7 


2.0 


-0.3 * 


0.071 


-14.3 


-0.26 


0.035 tt 


Service and volunteer: child report 


2.1 


1.8 


0.3 


0.356 


16.0 


0.23 


0.600 


Work for pay: parent report 


1.5 


1.6 


-0.1 


0.691 


-3.6 


-0.06 


0.252 


Work for pay: child report 


2.3 


1.7 


0.7 ** 


0.015 


39.7 


0.47 


0.378 


Service and work for pay s ummer 
















Service and volunteer: parent report 


1.7 


1.6 


0.1 


0.362 


9.2 


0.15 


0.581 


Service and volunteer: child report 


1.7 


1.6 


0.1 


0.706 


6.0 


0.08 


0.642 


Work for pay: parent report 


1.4 


1.6 


-0.2 


0.232 


-10.1 


-0.17 


0.175 


Work for pay: child report 


2.7 


1.5 


1.2 *** 


0.001 


81.3 


0.76 


0.002 ttt 


Parent approval of participation 


3.0 


2.9 


0.1 


0.565 


3.4 


0.09 


0.541 


Social activities - school year 
















Shop or eat out with adult: parent report 


3.2 


3.4 


-0.1 


0.270 


-4.2 


-0.17 


0.638 


Hang out with friends: parent report 


2.7 


2.9 


-0.1 


0.543 


-4.9 


-0.09 


0.467 


Shop or eat out with adult: child report 


3.4 


3.3 


0.1 


0.446 


3.8 


0.15 


0.779 


Hang out with friends: child report 


3.7 


3.6 


0.2 


0.531 


5.2 


0.12 


0.536 


Social activities - summer 
















Shop or eat out with adult: parent report 


3.3 


3.4 


-0.1 


0.411 


-3.1 


-0.12 


0.843 


Hang out with friends: parent report 


2.9 


2.8 


0.1 


0.742 


3.0 


0.05 


0.329 


Shop or eat out with adult: child report 


3.6 


3.4 


0.2 * 


0.096 


7.4 


0.27 


0.866 


Hang out with friends: child report 


3.9 


3.7 


0.2 


0.415 


6.4 


0.15 


0.764 


Television - school year 
















Watch TV (hours/day): parent report 


2.6 


2.6 


0.0 


0.724 


1.3 


0.05 


0.563 


Watch TV (hours/day): child report 


3.1 


2.9 


0.2 


0.150 


7.7 


0.29 


0.036 tt 


One potential barrier 
















Structured activities - school year 
Total structured activities: parent report 


2.4 


2.3 


0.1 


0.588 


2.6 


0.06 




Lessons 


2.0 


2.2 


-0.3 ♦ 


0.084 


-11.8 


-0.20 




Sports with a coach 


2.3 


2.4 


0.0 


0.845 


-1.5 


-0.02 




Club/youth group 


2.3 


2.3 


0.1 


0.695 


3.3 


0.05 




Religious class or activity 


3.0 


2.7 


0.3 ♦ 


0.064 


12.1 


0.24 




Community center 


2.4 


2.2 


0.2 


0.285 


9.5 


0.14 




School-related programs: parent report 
















Program to help with school 


2.0 


2.3 


-0.3 


— 0.184 


-11.6 


-0.17 




Before/after-school program 


2.0 


1.8 


0.1 


0.464 


8.2 


0.09 




Total structured activities: child report 


2.6 


2.6 


0.0 


0.819 


-1.1 


-0.03 




Lessons 


2.3 


2.6 


-0.2 


0.281 


-9.5 


-0.16 




Sports with a coach 


2.9 


2.7 


0.2 


0.371 


7.7 


0.13 




Club/youth group 


2.3 


2.5 


-0.2 


0.495 


-6.3 


-0.10 
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Appendix Table £.5.5 (continued) 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size^ 


P-Value for 
Difference 
Across 

Barrier Groups^ 


Religious class or activity 


2.9 


3.0 


0.0 


0.829 


-1.5 


-0.03 




Community center 


2.5 


2.4 


0.1 


0.636 


4.3 


0.06 




School-related programs: child report 


Program to help with school 


2.0 


2.1 


-0.1 


0.638 


-4.8 


-0.07 




Before/after-school program 


1.5 


1.4 


0.1 


0.707 


4.3 


0.05 




Structured activities - summer 


Total structured activities: parent report 


2.3 


2.2 


0.1 


0.529 


3.3 


0.07 




Lessons 


1.8 


1.7 


0.0 


0.868 


1.4 


0.02 




Sports with a coach 


2.1 


2.3 


-0.1 


0.437 


-6.5 


-0.11 




Club/youth group 


2.0 


2.1 


-0.1 


0.617 


-4.1 


-0.06 




Religious class or activity 


2.8 


2.5 


0.4 ** 


0.043 


14.5 


0.26 




Conununity center 


2.5 


2.3 


0.2 


0.350 


8.1 


0.12 




School-related programs: parent report 


Summer school 


2.2 


2.2 


0.0 


0.920 


-1.0 


-0.01 




Daycamp 


1.7 


1.9 


-0.3 


0.192 


-13.7 


-0.18 




Total structured activities: child report 


2.2 


2.1 


O.I 


0.492 


3.9 


0.10 




Lessons 


1.6 


1.6 


0.0 


0.973 


-0.4 


0.00 




Sports with a coach 


2.3 


2.1 


0.2 


0.392 


9.6 


0.13 




Club/youth group 


2.0 


1:8 


0.2 


0.449 


9.0 


0.12 




Religious class or activity 


2.9 


2.9 


O.I 


0.690 


2.8 


0.06 




Conununity center 


2.2 


2.2 


0.0 


0.837 


-2.2 


-0.03 




School-related programs: child report 


Summer school 


2.1 


2.1 


0.0 


0.988 


0.2 


0.00 




Daycamp 


1.8 


1.7 


0.1 


0.737 


4.6 


0.06 




Service and work for pay - school year 


Service and volunteer: parent report 


1.6 


1.5 


0.2 


0.110 


12.5 


0.17 




Service and volunteer: child report 


1.7 


1.7 


0.0 


0.931 


-1.2 


-0.02 




Work for pay: parent report 


1.4 


1.3 


0.1 


0.415 


6.4 


0.09 




Work for pay: child report 


2.0 


1.9 


0.1 


0.784 


4.0 


0.05 




Service and work for pay s ummer 


Service and volunteer: parent report 


1.5 


1.3 


0.2 * 


0.057 


16.5 


0.22 




Service and volunteer: child report 


1.5 


1.7 


-0.2 


0.370 


-12.5 


-0.18 




Work for pay: parent report 


1.4 


1.3 


0.1 


0.196 


10.6 


0.14 




Work for pay: child report 


2.2 


2.0 


0.2 


0.583 


9.7 


0.12 




Parent approval of participation 


2.7 


2.8 


-0.1 


0.583 


-2.8 


-0.07 




Social activities - school year 


Shop or eat out with adult: parent report 


3.4 


3.4 


0.0 


0.993 


0.0 


0.00 




Hang out with friends: parent report 


2.8 


2.9 


-0.2 


0.333 


-6.0 


-0.12 




Shop or eat out with adult: child report 


3.4 


3.4 


0.0 


0.911 


0.4 


0.02 




Hang out with friends: child report 


3.8 


3.6 


0.1 


0.495 


4.2 


0.10 
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Appendix Table E.5.5 (continued) 



P-Value for 
Difference 

Program Control P-Value for % Effect Across 



Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size® Barrier Groups^ 


Social activities - summer 














Shop or eat out with adult; parent report 


3.4 


3.6 


-0.2 * 


0.092 


-5.3 


-0.22 


Hang out with friends: parent report 


2.8 


2.9 


-0.1 


0.432 


-5.0 


-0.09 


Shop or eat out with adult: child report 


3.5 


3.3 


0.1 


0.323 


4.1 


0.15 


Hang out with friends: child report 


3.6 


3.6 


0.0 


0.913 


-0.7 


-0.02 


Television - school year 














Watch TV (hours/day): parent report 


2.6 


2.7 


-0.1 


0.401 


-2.7 


-0.11 


Watch TV (hours/day): child report 


3.1 


3.2 


-0.2 


0.192 


-4.8 


-0.20 


Two DOtential barriers or more 














Structured activities - school year 
Total structured activities: parent report 


2.5 


2.3 


0.2 


0.237 


8.8 


0.21 


Lessons 


2.2 


2.3 


-0.1 


0.604 


-5.1 


-0.09 


Sports with a coach 


2.4 


2.1 


0.3 


0.173 


15.5 


0.22 


Club/youth group 


2.5 


2.2 


0.3 


0.190 


14.6 


0.22 


Religious class or activity 


2.9 


2.6 


0.3 


0.191 


10.9 


0.21 


Community center 


2.6 


2.4 


0.2 


0.397 


9.6 


0.15 


School-related programs: parent report 














Program to help with school 


2.7 


2.3 


0.5 * 


0.068 


19.9 


0.30 


Before/after-school program 


2.5 


2.1 


0.4 


0.191 


16.9 


0.22 


Total structured activities: child report 


2.7 


2.6 


0.0 


0.752 


1.8 


0.06 


Lessons 


2.6 


2.4 


0.2 


0.436 


8.2 


0.13 


Sports with a coach 


2.7 


2.8 


-0.1 


0.613 


-5.0 


-0.09 


Club/youth group 


2.3 


2.2 


0.1 


0.810 


2.7 


0.04 


Religious class or activity 


3.2 


3.1 


0.1 


0.615 


4.7 


0.10 


Community center 


2.5 


2.6 


0.0 


0.927 


-1.0 


-0.02 


School-related programs: child report 














Program to help with school 


2.2 


2.0 


0.2 


0.458 


10.0 


0.13 


Before/after-school program 


1.5 


1.5 


0.1 


0.812 


3.7 


0.04 


Structured activities - summer 
Total structured activities: parent report 


2.4 


2.1 


0.2 


0.241 


10.3 


0.23 


Lessons 


1.8 


1.8 


0.1 


0.769 


3.5 


0.05 


Sports with a coach 


2.1 


1.9 


0.1 


0.568 


7.4 


0.10 


Club/youth group 


2.6 


2.1 


0.5 * 


0.062 


23.7 


0.34 


Religious class or activity 


2.9 


2.6 


0.4 


0.111 


14.1 


0.26 


Community center 


2.5 


2.4 


0.1 


0.841 


2.3 


0.04 


School-related programs: parent report 














Summer school 


2.0 


2.4 


-0.4 


0.136 


-16.2 


-0.23 


Daycamp 


1.7 


2.0 


-0.3 


0.191 


-15.8 


-0.22 


(continued) 




-240- 



276 



Appendix Table £.5.5 (continued) 



P- Value for 
Difference 

Program Control P-Value for % Effect Across 



Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size^ Barrier Groups 


Total structured activities: child report 


2.4 


2.2 


0.2 


0.206 


7.0 


0.19 


Lessons 


1.7 


1.7 


0.0 


0.902 


1.4 


0.02 


Sports with a coach 


2.5 


2.4 


0.2 


0.535 


6.7 


0.10 


Club/youth group 


2.0 


1.8 


0.2 


0.418 


9.5 


0.13 


Religious class or activity 


3.2 


2.7 


0.5 ♦ 


0.063 


17.8 


0.34 


Community center 


2.5 


2.6 


-0.1 


0.819 


-2.6 


-0.04 


School-related programs: child report 


Summer school 


1.9 


2.6 


-0.7 ♦♦ 


0.016 


-25.5 


-0.39 


Daycamp 


1.7 


1.8 


-0.1 


0.656 


-6.6 


-0.08 


Service and work for pay - school year 


Service and volunteer: parent report 


1.8 


1.6 


0.2 


0.205 


11.9 


0.18 


Service and volunteer: child report 


1.7 


1.8 


-0.2 


0.534 


-9.5 


-0.14 


Work for pay: parent report 


1.5 


1.3 


0.3 * 


0.059 


20.2 


0.27 


Work for pay: child report 


2.0 


1.8 


0.2 


0.508 


11.4 


0.15 


Service and work for pay - summer 


Service and volunteer: parent report 


1.7 


1.4 


0.3 ** 


0.038 


24.5 


0.35 


Service and volunteer: child report 


1.7 


1.7 


0.1 


0.864 


4.3 


0.06 


Work for pay: parent report 


1.4 


1.3 


0.1 


0.286 


10.7 


0.15 


Work for pay: child report 


1.8 


2.4 


-0.6 * 


0.078 


-25.3 


-0.38 


Parent approval of participation 


2.8 


2.7 


0.1 


0.500 


4.5 


0.10 


Social activities - school year 


Shop or eat out with adult: parent report 


3.3 


3.4 


0.0 


0.719 


-1.4 


-0.06 


Hang out with friends: parent report 


3.0 


2.9 


0.1 


0.613 


3.7 


0.07 


Shop or eat out with adult: child report 


3.5 


3.4 


0.1 


0.316 


3.9 


0.16 


Hang out with friends: child report 


3.8 


4.0 


-0.2 


0.489 


-4.4 


-0.11 


Social activities - summer 


Shop or eat out with adult: parent report 


3.3 


3.5 


-0.2 


0.147 


-5.5 


-0.22 


Hang out with friends: parent report 


3.1 


2.9 


0.2 


0.295 


7.9 


0.14 


Shop or eat out with adult: child report 


3.4 


3.2 


0.2 


0.222 


6.4 


0.22 


Hang out with friends: child report 


4.0 


4.0 


0.0 


0.931 


0.6 


0.01 


Television - school year 


Watch TV (hours/day): parent report 


2.8 


2.9 


-0.1 


0.394 


-3.3 


-0.14 


Watch TV (hours/day): child report 


3.0 


3.2 


1 

p 

k) 

« 


0.061 


-7.2 


-0.30 
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Appendix Table £.5.5 (continued) 

SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Table 5.6 presents the ranges of outcomes reported in this table. 

For the group with no potential barriers, parent reports were available for 222 children; and child reports were 
available for 222 children. For the group with one potential barrier, parent reports were available for 345 children; and 
child reports were available for 345 children. For the group with two potential barriers or more, parent reports were 
available for 263 children; and child reports were available for 263 children. Actual sample sizes for individual measures 
may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ftt ^ percent, ft = 5 percent, and 
t = 10 percent. 
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The New Hope Project 
Appendix Table £.5.6 

Impacts on Children's Activities, by Ethnicity 



P-Value for 
Difference 



Outcome 


Program Control 
Group Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size" 


Across 

Ethnicities^ 


African-American 
Structured activities - school year 


Total structured activities: parent report 


2.5 


2.5 


0.0 


0.924 


0.4 


0.01 


0.714 


Lessons 


2.1 


2.3 


-0.2 * 


0.095 


-10.2 


-0.18 


0.850 


Sports with a coach 


2.2 


2.3 


0.0 


0.826 


-1.4 


-0.02 


0.567 


Club/youth group 


2.5 


2.4 


0.0 


0.839 


1.3 


0.02 


0.458 


Religious class or activity 


3.2 


2.9 


0.2 


0.106 


8.1 


0.18 


0.542 


Community center 


2.6 


2.5 


0.1 


0.676 


2.8 


0.05 


0.783 


School-related programs; parent report 


Program to help with school 


2.5 


2.3 


0.2 


0.239 


9.0 


0.13 


0.121 


Before/after-school program 


2.2 


2.1 


0.1 


0.565 


4.9 


0.06 


0.406 


Total structured activities: child report 


2.7 


2.7 


0.0 


0.894 


0.5 


0.02 


0.343 


Lessons 


2.5 


2.5 


-0.1 


0.733 


-2.6 


-0.04 


0.700 


Sports with a coach 


2.8 


2.9 


-0.1 


0.510 


-4.4 


-0.08 


0.567 


Club/youth group 


2.4 


2.4 


0.0 


0.922 


-0.8 


-0.01 


0.669 


Religious class or activity 


3.2 


3.0 


0.2 


0.180 


8.2 


0.18 


0.320 


Community center 


2.6 


2.6 


0.0 


0.866 


1.3 


0.02 


0.587 


School-related programs: child report 


Program to help with school 


2.2 


2.1 


0.1 


0.602 


4.8 


0.07 


0.241 


Before/after-school program 


1.6 


1.6 


0.0 


0.976 


0.3 


0.00 


0.614 


Structured activities - summer 


Total structured activities: parent report 


2.4 


2.4 


0.0 


0.831 


1.1 


0.03 


0.274 


Lessons 


1.8 


1.9 


-0.1 


0.383 


-6.4 


-0.10 


0.220 


Sports with a coach 


2.0 


2.2 


-0.2 


0.219 


-8.8 


-0.13 


0.602 


Club/youth group 


2.4 


2.3 


0.1 


0.632 


3.5 


0.06 


0.571 


Religious class or activity 


3.1 


2.8 


0.4 ♦♦ 


0.021 


13.0 


0.26 


0.184 


Community center 


2.6 


2.7 


0.0 


0.827 


-1.5 


-0.03 


0.167 


School-related programs: parent report 


Summer school 


2.2 


2.4 


-0.3 


0.176 


-10.9 


-0.16 


0.545 


Daycamp 


1.8 


2.2 


-0.4 ♦♦ 


0.022 


-19.0 


-0.28 


0.062 t 


Total structured activities: child report 


2.3 


2.2 


0.0 


0.636 


2.1 


0.06 


0.466 


Lessons 


1.6 


1.6 


0.0 


0.919 


-0.9 


-0.01 


0.980 


Sports with a coach 


2.2 


2.3 


-0.1 


0.740 


-2.7 


-0.04 


0.896 


Club/youth group 


2.0 


1.8 


0.2 


0.306 


10.0 


0.13 


0.122 


Religious class or activity 


3.2 


2.9 


0.3 


0.120 


9.8 


0.20 


0.994 


Community center 


2.3 


2.5 


-0.2 


0.440 


-6.3 


-0.10 


0.510 
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Appendix Table £.5.6 (continued) 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size" 


P-Value for 
Difference 
Across 
Ethnicities^ 


School-related programs: child report 


Summer school 


2.2 


2.2 


0.0 


0.827 


-2.0 


-0.03 


0.006 ttt 


Daycamp 


1.7 


1.8 


-0.2 


0.407 


-8.6 


-0.11 


0.507 


Service and work for pay > school year 


Service and volunteer: parent report 


1.7 


1.7 


0.0 


0.936 


0.5 


0.01 


0.742 


Service and volunteer: child report 


1.9 


1.8 


0.1 


0.590 


6.5 


0.09 


0.047 tt 


Work for pay: parent report 


1.5 


1.4 


0.1 


0.176 


9.4 


0.14 


0.905 


Work for pay: child report 


2.2 


2.0 


0.2 


0.373 


10.3 


0.15 


0.767 


Service and work for pay - summer 


Service and volunteer: parent report 


1.7 


1.5 


0.2 ** 


0.049 


15.1 


0.23 


0.707 


Service and volunteer child report 


1.7 


1.8 


-0.2 


0.469 


-8.6 


-0.13 


0.453 


Work for pay: parent report 


1.5 


1.4 


0.1 


0.587 


3.6 


0.05 


0.988 


Work for pay: child report 


2.2 


2.1 


0.1 


0.742 


4.3 


0.06 


0.145 


Parent approval of participation 


2.8 


3.0 


-0.2 * 


0.092 


-6.8 


-0.17 


0.008 ttt 


Social activities - school year 


Shop or eat out with adult: parent report 


3.3 


3.2 


0.1 


0.451 


2.3 


0.09 


0.019 tt 


Hang out with friends: parent report 


2.9 


2.9 


0.0 


0.982 


-0.1 


0.00 


0.073 t 


Shop or eat out with adult: child report 


3.3 


3.3 


-0.1 


0.454 


-2.4 


-0.10 


0.176 


Hang out with friends: child report 


3.8 


3.8 


0.0 


0.896 


0.6 


0.02 


0.956 


Social activities > summer 


Shop or eat out with adult: parent report 


3.2 


3.4 


-0.1 


0.237 


-3.5 


-0.14 


0.222 


Hang out with friends: parent report 


3.0 


2.9 


0.0 


0.780 


1.5 


0.03 


0.248 


Shop or eat out with adult: child report 


3.4 


3.2 


0.1 


0.215 


4.5 


0.16 


0.773 


Hang out with friends: child report 


3.9 


3.9 


0.0 


0.993 


0.0 


0.00 


0.474 


Television - school year 


Watch TV (hours/day): parent report 


2.7 


2.9 


-0.1 * 


0.059 


-5.1 


-0.22 


0.098 t 


Watch TV (hours/day): child report 


3.2 


3.2 


-0.1 


0.442 


-2.2 


-0.09 


0.067 t 


Hisnanic 

Structured activities - school year 


Total structured activities: parent report 


2.3 


2.2 


0.1 


0.518 


4.2 


0.10 




Lessons 


1.9 


2.1 


-0.2 


0.313 


-9.0 


-0.14 




Sports with a coach 


2.4 


2.3 


0.1 


0.715 


3.8 


0.06 




Club/youth group 


2.2 


2.0 


0.3 


0.245 


13.1 


0.18 




Religious class or activity 


2.7 


2.6 


0.1 


0.701 


3.6 


0.07 




Community center 


2.1 


1.9 


0.2 


0.364 


11.8 


0.15 




School-related programs: parent report 


Program to help with school 


2.0 


2.4 


-0.3 


0.167 


-14.4 


-0.22 




Before/after-school program 


2.0 


1.9 


0.1 


0.753 


4.7 


0.05 
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Appendix Table £.5.6 (continued) 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% 

Impact 


P-Value for 
Difference 
Effect Across 
Size® Ethnicities^ 


Total structured activities: child report 


2.4 


2.6 


-0.2 


0.280 


-7.6 


-0.24 


Lessons 


2.4 


2.4 


0.0 


0.996 


-0.1 


0.00 


Sports with a coach 


2.6 


2.9 


-0.3 


0.282 


-11.2 


-0.20 


Club/youth group 


2.1 


2.3 


-0.3 


0.370 


-11.8 


-0.18 


Religious class or activity 


3.1 


3.2 


-0.1 


0.724 


-3.1 


-0.07 


Community center 


2.0 


2.3 


-0.3 


0.315 


-12.6 


-0.19 


School-related programs: child report 














Program to help with school 


1.8 


2.3 


-0.5 ♦ 


0.078 


-20.9 


-0.32 


Before/after-school program 


1.4 


1.4 


0.0 


0.849 


-3.2 


-0.04 


Structured activities - summer 














Total structured activities: parent report 


2.0 


2.0 


0.0 


0.998 


0.0 


0.00 


Lessons 


1.6 


1.6 


0.0 


0.791 


2.6 


0.03 


Sports with a coach 


2.0 


2.0 


0.0 


0.836 


2.5 


0.04 


Club/youth group 


1.9 


2.0 


-0.1 


0.569 


-6.2 


-0.09 


Religious class or activity 


2.5 


2.5 


0.0 


0.967 


0.4 


0.01 


Community center 


2.0 


2.0 


0.0 


0.977 


0.4 


0.00 


School-related programs: parent report 














Summer school 


2.1 


2.2 


-0.1 


0.754 


-4.3 


-0.06 


Daycamp 


1.8 


1.7 


0.1 


0.688 


6.0 


0.07 


Total structured activities: child report 


1.9 


2.1 


-0.2 


0.298 


-8.1 


-0.21 


Lessons 


1.6 


1.6 


0.0 


0.938 


-1.2 


-0.02 


Sports with a coach 


2.0 


2.1 


-0.1 


0.651 


-5.8 


-0.08 


Club/youth group 


1.5 


1.9 


-0.4 ♦ 


0.093 


-20.8 


-0.28 


Religious class or activity 


2.9 


2.6 


0.3 


0.364 


10.1 


0.19 


Community center 


1.8 


2.3 


-0.5 * 


0.095 


-21.8 


-0.31 


School-related programs: child report 














Summer school 


1.5 


2.7 


-1.2 ♦♦♦ 


0.000 


-45.1 


-0.73 


Daycamp 


1.8 


1.7 


0.1 


0.694 


7.7 


0.09 


Service and work for pay - school year 














Service and volunteer: parent report 


1.7 


1.5 


0.1 


0.353 


8.6 


0.12 


Service and volunteer: child report 


1.3 


1.9 


-0.6 ♦♦ 


0.019 


-33.0 


-0.50 


Work for pay: parent report 


1.3 


1.2 


0.1 


0.483 


5.9 


0.08 


Work for pay: child report 


1.8 


1.7 


0.1 


0.778 


4.8 


0.06 


Service and work for pay - summer 














Service and volunteer: parent report 


1.6 


1.4 


0.2 ♦ 


0.089 


17.3 


0.24 


Service and volunteer: child report 


1.3 


1.7 


-0.3 


0.289 


-19.4 


-0.28 


Work for pay: parent report 


1.3 


1.2 


0.1 


0.597 


4.9 


0.06 


Work for pay: child report 


2.3 


1.7 


0.5 


0.202 


29.7 


0.32 
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Appendix Table £.5.6 (continued) 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% 

Impact 


P-Value for 
Difference 
Effect Across 
Size^ Ethnicities^ 


Parent approval of participation 


2.8 


2.5 


0.4 * 


0.053 


15.1 


0.32 


Social activities - school year 


Shop or eat out with adult: parent report 


3.4 


3.6 


-0.3 ♦ 


0.070 


-7.1 


-0.31 


Hang out with friends: parent report 


2.2 


2.6 


-0.4 ♦ 


0.061 


-16.5 


-0.29 


Shop or eat out with adult: child report 


3.6 


3.5 


0.2 


0.294 


4.7 


0.19 


Hang out with friends: child report 


3.5 


3.4 


0.0 


0.847 


1.4 


0.03 


Social activities - summer 


Shop or eat out with adult: parent report 


3,4 


3.7 


-0.3 ♦ 


0,052 


-7,2 


-0.31 


Hang out with friends: parent report 


2.2 


2.6 


-0.3 


0.170 


-13.2 


-0.22 


Shop or eat out with adult: child report 


3,7 


3,4 


0,3 ♦ 


0,053 


8.3 


0.31 


Hang out with friends: child report 


3,6 


3.2 


0,4 


0,200 


12.8 


0.26 


Television - school year 


Watch TV (hours/day): parent report 


2.5 


2,4 


0,1 


0,383 


3,3 


0,12 


Watch TV (hours/day): child report 


3.0 


2.9 


0.0 


0.739 


1,6 


0.06 


White 

Structured activities - school year 


Total structured activities: parent report 


2,2 


2.0 


0.2 


0.364 


8,0 


0.17 


Lessons 


2,0 


2.0 


-0.1 


0.800 


-3,1 


-0.05 


Sports with a coach 


2,4 


2.1 


0.3 


0.308 


15.7 


0.22 


Club/youth group 


1,7 


1.9 


-0.1 


0.558 


-7.8 


-0.10 


Religious class or activity 


2,4 


2.0 


0.5 


0.140 


23.6 


0.34 


Community center 


2.3 


2.1 


0.2 


0,491 


10.2 


0.14 


School -related programs: parent report 


Program to help with school 


2,0 


1,9 


0.2 


0,568 


8,3 


0.10 


Before/after-school program 


2.0 


1.4 


0.6 


0,130 


39,4 


0.34 


Total structured activities: child report 


2,6 


2,3 


0.3 


0,322 


12,1 


0.33 


Lessons 


2.9 


2.5 


0.5 


0,322 


18.2 


0.29 


Sports with a coach 


2.4 


2.1 


0.3 


0,567 


16,2 


0,21 


Club/youth group 


2.3 


2.1 


0.2 


0.676 


10.1 


0,14 


Religious class or activity 


3.0 


2,3 


0,6 


0.153 


27.9 


0,46 


Community center 


2.2 


2.5 


-0.3 


0.519 


-10.6 


-0,17 


School-related programs: child report 


Program to help with school 


1.8 


1.8 


-0,1 


0.876 


-3.7 


-0,04 


Before/after-school program 


1.7 


1.2 


0.4 


0.294 


35,5 


0,36 
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Appendix Table £.5.6 (continued) 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% 

Impact 


P-Value for 
Difference 
Effect Across 
Size® Ethnicities^ 


Structured activities - summer 


Total structured activities: parent report 


2.0 


1.7 


0.3 * 


0.063 


18.4 


0.32 


Lessons 


1.7 


1.4 


0.3 


0.137 


23.9 


0.27 


Sports with a coach 


2.0 


2.0 


-0.1 


0.816 


-3.9 


-0.06 


Club/youth group 


1.9 


1.7 


0.2 


0.502 


11.5 


0.14 


Religious class or activity 


2.4 


1.9 


0.6 * 


0.067 


30.0 


0.41 


Community center 


2.1 


1.6 


0.6 


0.116 


36.0 


0.37 


School-related programs: parent report 


Summer school 


2.1 


2.0 


0.2 


0.742 


8.0 


0.09 


Daycamp 


1.3 


1.3 


0.0 


0.911 


1.9 


0.02 


Total structured activities: child report 


2.0 


2.0 


0.0 


0.965 


-0.5 


-0.01 


Lessons 


1.4 


1.5 


-0.1 


0.749 


-6.7 


-0.08 


Sports with a coach 


2.4 


2.2 


0.2 


0.740 


9.4 


0.13 


Club/youth group 


1.8 


1.6 


0.2 


0.553 


15.7 


0.18 


Religious class or activity 


2.6 


2.4 


0.2 


0.690 


9.8 


0.16 


Community center 


2.0 


2.6 


-0.6 


0.191 


-23.6 


-0.38 


School-related programs: child report 


Summer school 


1,5 


2.1 


-0.6 


0.179 


-27.6 


-0.34 


Daycamp 


1.7 


1.4 


0.3 


0.558 


19.7 


0.19 


Service and work for pay - school year 


Service and volunteer: parent report 


1,6 


1.5 


0.1 


0.610 


8.2 


0.11 


Service and volunteer: child report 


2.4 


1.8 


0.6 


0.316 


30.0 


0.43 


Work for pay: parent report 


1,6 


1.5 


0.1 


0.798 


3.9 


0.06 


Work for pay: child report 


1,6 


2.0 


-0.3 


0.604 


-17.0 


-0,24 


Service and work for pay - summer 


Service and volunteer: parent report 


1,3 


1,2 


0.1 


0.552 


7.8 


0.10 


Service and volunteer: child report 


1,7 


1.3 


0.4 


0.399 


33.6 


0.37 


Work for pay: parent report 


1,6 


1.5 


0.1 


0.607 


5.8 


0.09 


Work for pay: child report 


1.7 


2.8 


-1.1 * 


0.091 


-40.1 


-0.71 


Parent approval of participation 


3,0 


2.7 


0.3 


0.241 


10.8 


0.25 


Social activities - school year 


Shop or eat out with adult: parent report 


3,7 


3.5 


0.2 


0.126 


6.8 


0.29 


Hang out with friends: parent report 


3,6 


3,3 


0.4 


0.309 


10.8 


0.24 


Shop or eat out with adult: child report 


3,6 


3.2 


0,5 


0.174 


14.7 


0.56 


Hang out with friends: child report 


4,0 


3.8 


0.2 


0.739 


5.9 


0.14 


Social activities - summer 


Shop or eat out with adult: parent report 


3,7 


3,6 


0.1 


0.599 


2.9 


0.12 


Hang out with friends: parent report 


3,5 


3.4 


0.2 


0.608 


5.4 


0.11 


Shop or eat out with adult: child report 


3,6 


3.4 


0.1 


0.678 


4.2 


0.15 


Hang out with friends: child report 


4,0 


3,8 


0,2 


0.802 


4.3 


0.10 
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Appendix Table £.5.6 (continued) 



Outcome 


Program Control 
Group Group 


P-Value for 
Difference Difference 


% 

Impact 


Effect 

Size“ 


P-Value for 
Difference 
Across 
Ethnicities'* 


Television - school year 

Watch TV (hours/day): parent report 
Watch TV (hours/day): child report 


2.5 2.5 

2.6 3.3 


0.0 0.827 

-0.7 *** 0.009 


-1.0 

-21.4 


-0.04 

-0.92 





SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 5.6 presents the ranges of outcomes reported in this table. 

For African-Americans, parent reports were available for 462 children; and child reports were available for 462 
children. For Hispanics, parent reports were available for 236 children; and child reports were available for 236 children. 
For whites, parent reports were available for 109 children; and child reports were available for 109 children. Actual sample 
sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control groip outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and 
t = 1 0 percent. 
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The New Hope Project 
Appendix Table E.6,1 

Impacts on Children's Achievement and School Progress, by Age 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size® 


P- Value for 
Difference 
Across 
Age Groups^ 


Aees 6-8 

Woodcock-Johnson test of achievement 


Total standard score 


102.0 


99.2 


2.9 


0.136 


2.9 


0.19 


0.737 


Broad reading score 


103.0 


100.3 


2.8 


0.133 


2.7 


0.17 


0.714 


Letter-word score 


100.9 


96.8 


4.1 * 


0.051 


4.2 


0.21 


0.606 


Comprehension 


105.8 


104.3 


1.4 


0.452 


1.4 


0.09 


0.763 


Broad math score 


101.0 


98.2 


2.8 


0.230 


2.9 


0.17 


0.649 


Calculation 


102.8 


100.3 


2.5 


0.257 


2.5 


0.13 


0.881 


Applied problems 


100.3 


97.7 


2.7 


0.345 


2.7 


0.16 


0.638 


Parent ratings of achievement 


Overall achievement 


3.9 


4.0 


-0.1 


0.396 


-2.7 


-0.10 


0.085 t 


Reading 


3.8 


3.5 


0.2 


0.131 


6.3 


0.19 


0.356 


Math 


3.9 


3.8 


O.I 


0.347 


2.8 


0.10 


0.301 


Teacher ratings of achievement 
Social Skills Rating System academic 


subscale 


3.3 


3.3 


0.0 


0.825 


-1.0 


-0.03 


0.306 


Mock report card - total 


3.0 


3.1 


0.0 


0.808 


-1.2 


-0.04 


0.870 


Mock report card - reading 


3.0 


3.0 


0.0 


0.982 


0.1 


0.00 


0.411 


Mock report card - math 


3.0 


3.1 


-0.1 


0.668 


-2.2 


-0.06 


0.769 


Classroom Behavior Scale 


3.7 


3.7 


0.0 


0.828 


-0.8 


-0.03 


0.694 


Teacher expectations for child 


3.0 


3.1 


0.0 


0.779 


-1.5 


-0.04 


0.779 


School progress 


Positive school progress: parent report 


0.4 


0.4 


0.0 


0.336 


9.6 


0.12 


0.973 


Receives gifted services: teacher report 


1.1 


1.1 


0.0 


0.505 


4.2 


0.10 


0.616 


Negative school progress: parent report 


0.1 


0.1 


0.0 


0.823 


5.0 


0.02 


0.120 


Receives remedial services: 


teacher report 


1.4 


1.3 


0.1 


0.341 


7.3 


0.14 


0.367 


Time absent: teacher report 


1.7 


1.7 


0.0 


0.918 


-0.9 


-0.01 


0.801 


Time tardy: teacher report 


1.6 


1.6 


0.0 


0.837 


-2.3 


-0.03 


0.644 


Aees 9-12 

Woodcock-Johnson test of achievement 


Total standard score 


95.2 


93.9 


1.4 


0.426 


1.4 


0.09 




Broad reading score 


97.6 


95.3 


2.4 


0.234 


2.5 


0.14 




Letter-word score 


98.0 


95.1 


2.8 


0.251 


3.0 


0.14 
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Appendix Table E.6.1 (continued) 



Program Control P-Value for % Effect 



Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size® 


Comprehension 


97.3 


95.4 


1.9 


0.319 


2.0 


0.12 


Broad math score 


92.8 


92.5 


0.2 


0.892 


0.3 


0.01 


Calculation 


86.6 


85.3 


1.3 


0.514 


1.6 


0.07 


Applied problems 


99.0 


99.4 


-0.3 


0.855 


-0.3 


-0.02 


Parent ratings of achievement 


Overall achievement 


3.7 


3.8 


-0.1 


0.425 


-2.6 


-0.09 


Reading 


3.6 


3.5 


0.1 


0.633 


1.8 


0.05 


Math 


3.6 


3.7 


-0.1 


0.424 


-2.6 


-0.09 


Teacher ratings of achievement 
Social Skills Rating System academic 


subscale 


3.1 


3.2 


0.0 


0.759 


-1.5 


-0.05 


Mock report card - total 


2.8 


2.8 


-0.1 


0.728 


-1.9 


-0.06 


Mock report card - reading 


2.8 


2.8 


-0.1 


0.702 


-2.3 


-0.06 


Mock report card - math 


2.7 


2.8 


-0.1 


0.694 


-2.5 


-0.06 


Classroom Behavior Scale 


3.7 


3.8 


0.0 


0.830 


-0.9 


-0.03 


Teacher expectations for child 


2.8 


2.8 


-0.1 


0.747 


-2.0 


-0.05 


School progress 


Positive school progress: parent report 


0.5 


0.4 


0.0 


0.328 


8.3 


0.11 


Receives gifted services: teacher report 


1.1 


1.2 


-0.1 


0.449 


-4.8 


-0.13 


Negative school progress: parent report 


0.2 


0.2 


0.0 


0.680 


7.3 


0.05 


Receives remedial services: 


teacher report 


1.4 


1.5 


-0.1 


0.361 


-6.7 


-0.16 


Time absent: teacher report 


1.7 


1.6 


0.1 


0.430 


7.5 


0.11 


Time tardy: teacher report 


1.4 


1.5 


-0.2 


0.235 


-10.8 


-0.15 


Aees 13-16 

Woodcock-Johnson test of achievement 


Total standard score 


88.8 


88.0 


0.9 


0.657 


1.0 


0.06 


Broad reading score 


91.8 


91.6 


0.2 


0.935 


0.2 


0.01 


Letter -word score 


92.2 


92.2 


0.1 


0.986 


0.1 


0.00 


Comprehension 


90.8 


91.2 


-0.3 


0.895 


-0.4 


-0.02 


Broad math score 


85.9 


84.4 


1.5 


0.412 


1.8 


0.09 


Calculation 


79.1 


76.4 


2.7 


0.236 


3.5 


0.14 


Applied problems 


93.0 


92.3 


0.6 


0.726 


0.7 


0.04 


Parent ratings of achievement 


Overall achievement 


3.6 


3.4 


0.3 ♦ 


0.067 


8.4 


0.26 


Reading 


3.7 


3.4 


0.3 ♦♦ 


0.019 


10.4 


0.30 


Math 


3.5 


3.3 


0.2 


0.226 


5.5 


0.17 



P- Value for 
Difference 
Across 
Age Groups^ 
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Appendix Table E.6.1 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size^ 


P- Value for 
Difference 
Across 
Age Groups^ 


Teacher ratings of achievement 
Social Skills Rating System academic 
subscale 


3.2 


2.9 


0.3 


0.135 


10.3 


0.30 




Mock report card - total 


2.8 


2.7 


0.1 


0.718 


3.5 


0.09 




Mock report card - reading 


2.9 


2.6 


0.3 


0.210 


11.7 


0.28 




Mock report card - math 


2.6 


2.5 


0.1 


0.614 


5.3 


0.12 




Classroom Behavior Scale 


3.7 


3.5 


0.2 


0.420 


4.8 


0.16 




Teacher expectations for child 


2.7 


2.5 


0.1 


0.564 


5.2 


0.12 




School progress 


Positive school progress: parent report 


0.4 


0.3 


0.0 


0.551 


7.5 


0.08 




Receives gifted services: teacher report 


1.1 


1.1 


0.0 


0.881 


-1.0 


-0.02 




Negative school progress: parent report 


0.2 


0.3 


-0.1 ** 


0.035 


-25.7 


-0.30 




Receives remedial services: 
teacher report 


1.4 


1.5 


0.0 


0.805 


-2.3 


-0.05 




Time absent: teacher report 


2.5 


2.5 


0.1 


0.791 


2.7 


0.06 




Time tardy: teacher report 


2.0 


1.9 


0.1 


0.714 


4.7 


0.08 





SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 6.1 presents the ranges of outcomes reported in this table. 

For ages 6 to 8, test results were available for 279 children; parent reports were available for 279 children; and teacher 
reports were available for 191 children. For ages 9 to 12, test results were available for 303 children; parent reports were 
available for 303 children; and teacher reports were available for 193 children. For ages 13 to 16, test results were 
available for 234 children; parent reports were available for 234 children; and teacher reports were available for 142 
children. Actual sample sizes for individual measures may vary as a result of missing data. 

*^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 1 0 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and t = 

1 0 percent. 
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Appendix Table E.6.2 

Impacts on Children's Achievement and School Progress, by Number of Parent's 
Potential Barriers to Employment 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size“ 


P-Value for 
Difference 
Across 

Barrier Groups^ 


No DOtential barriers 

Woodcock- Johnson test of achievement 


Total standard score 


95.1 


96.9 


-1.8 


0.446 


-1.8 


-0.12 


0.104 


Broad reading score 


98.1 


97.6 


0.6 


0.814 


0.6 


0.03 


0.414 


Letter-word score 


98.0 


96.0 


2.0 


0.445 


2.1 


0.10 


0.645 


Comprehension 


97.7 


99.5 


-1.8 


0.521 


-1.8 


-0.11 


0.148 


Broad math score 


93.5 


94.4 


-0.8 


0.736 


-0.9 


-0.05 


0.469 


Calculation 


88.9 


88.7 


0.3 


0.926 


0.3 


0.01 


0.751 


Applied problems 


98.2 


101.0 


-2.8 


0.288 


-2.8 


-0.17 


0.074 t 


Parent ratings of achievement 


Overall achievement 


3.8 


4.0 


-0.2 


0.201 


-4.2 


-0.16 


0.412 


Reading 


3.9 


3.8 


0.1 


0.744 


1.4 


0.04 


0.352 


Math 


3.8 


3.8 


0.1 


0.559 


2.3 


0.08 


0.737 


Teacher ratings of achievement 
Social Skills Rating System academic 


subscale 


3.3 


3.1 


0.2 


0.369 


5.8 


0.18 


0.328 


Mock report card - total 


3.0 


2.9 


0.0 


0.863 


1.2 


0.04 


0.358 


Mock report card - reading 


3.0 


2.9 


0.1 


0.735 


2.3 


0.06 


0.073 t 


Mock report card - math 


2.8 


3.0 


-0.2 


0.386 


-6.6 


-0.18 


0.729 


Classroom Behavior Scale 


3.7 


3.6 


0.0 


0.861 


1.0 


0.03 


0.605 


Teacher expectations for child 


3.1 


3.0 


0.1 


0.739 


2.6 


0.07 


0.235 


School progress 


Positive school progress: parent report 


0.4 


0.4 


0.1 


0.119 


20.1 


0.23 


0.712 


Receives gifted services: teacher report 


1.2 


1.1 


0.0 


0.831 


1.9 


0.05 


0.976 


Negative school progress: parent report 


0.2 


0.2 


0.0 


0.461 


17.8 


0.10 


0.360 


Receives remedial services: 


teacher report 


1.4 


1.4 


0.0 


0.997 


0.0 


0.00 


0.655 


Time absent: teacher report 


1.7 


2.0 


-0.3 


0.153 


-15.3 


-0.28 


0.106 


Time tardy: teacher report 


1.5 


1.8 


-0.2 


0.276 


-13.7 


-0.22 


0.261 


One potential barrier 
Woodcock-Johnson test of achievement 


Total standard score 


97.8 


96.4 


1.4 


0.376 


1.5 


0.10 




Broad reading score 


99.1 


98.1 


1.0 


0.572 


1.1 


0.06 




Letter-word score 


98.9 


97.0 


1.9 


0.438 


2.0 


0.10 





(continued) 




-252- 



288 



Appendix Table E.6.2 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size® 


P-Value for 
Difference 
Across 

Barrier Groups^ 


Comprehension 


99,5 


99,7 


-0,1 


0,932 


-0,2 


-0,01 




Broad math score 


95.2 


93,5 


1,7 


0,335 


1,8 


0,10 




Calculation 


92,5 


90,0 


2,4 


0,190 


2,7 


0,13 




Applied problems 


98,2 


97,7 


0,4 


0,823 


0,5 


0,03 




Parent ratings of achievement 


Overall achievement 


3.7 


3.7 


0,0 


0,771 


1,0 


0,03 




Reading 


3,8 


3,4 


0,3 


0,006 


10,2 


0,30 




Math 


3.7 


3,7 


0,0 


0,778 


-0,9 


-0,03 




Teacher ratings of achievement 
Social Skills Rating System academic 


subscale 


3.4 


3.2 


0,2 * 


0,083 


7.3 


0,23 




Mock report card - total 


3.1 


2.8 


0.2 


0,104 


7,8 


0,22 




Mock report card - reading 


3.1 


2.7 


0.4 ** 


0.016 


13,5 


0,34 




Mock report card - math 


2.9 


2.9 


0.0 


0.782 


1,5 


0,04 




Classroom Behavior Scale 


3.9 


3.7 


0.2 


0.253 


4,3 


0,15 




Teacher expectations for child 


3.0 


2.8 


0.1 


0.414 


4.6 


0,11 




School progress 


Positive school progress: parent report 


0.4 


0.4 


0.0 


0.281 


9,8 


0,12 




Receives gifted services: teacher report 


1.2 


1.1 


0.0 


0.483 


4.1 


0,10 




Negative school progress: parent report 


0.2 


0.2 


0.0 


0.632 


-6,8 


-0,05 




Receives remedial services: 


teacher report 


1.4 


1.5 


-0.1 


0.311 


-6.7 


-0,15 




Time absent: teacher report 


1.9 


1.8 


0.0 


0.776 


2.3 


0.04 




Time tardy: teacher report 


1.6 


1.6 


-0.1 


0.776 


-3.1 


-0.05 




Two potential barriers or more 
Woodcock>Johnson test of achievement 


Total standard score 


94.9 


90.6 


4.4 ** 


0.036 


4.8 


0.30 




Broad reading score 


96.4 


92.0 


4.4 * 


0.098 


4.7 


0.27 




Letter-word score 


95.6 


90.7 


5.0 


0.115 


5.5 


0.25 




Comprehension 


97.5 


93.3 


4.2 * 


0.087 


4.5 


0.26 




Broad math score 


92.4 


89.4 


3.0 


0.167 


3.4 


0.18 




Calculation 


88.4 


85.9 


2.6 


0.206 


3.0 


0.13 




Applied problems 


97.3 


92.3 


5.0 * 


0.056 


5.4 


0.29 




Parent ratings of achievement 


Overall achievement 


3.7 


3.6 


0.1 


0.558 


2.4 


0.08 




Reading 


3.5 


3.3 


0.2 


0.160 


6.8 


0.19 




Math 


3.6 


3.5 


0.1 


0.529 


2.6 


0.08 
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P-Value for 
Difference 




Program 


Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size“ 


Barrier Groups® 


Teacher ratings of achievement 
















Social Skills Rating System academic 
subscale 


3.0 


3.1 


-0.1 


0.629 


-3.0 


-0.09 




Mock report card - total 


2.7 


2.8 


-0.1 


0.586 


-3.7 


-0.11 




Mock report card - reading 


2.6 


2.8 


-0.2 


0.347 


-7.2 


-0.18 




Mock report card - math 


2.7 


2.7 


0.0 


0.986 


0.1 


0.00 




Classroom Behavior Scale 


3.5 


3.6 


-0.1 


0.706 


-1.6 


-0.06 




Teacher expectations for child 


2.4 


2.7 


-0.3 


0.188 


-9.7 


-0.23 




School progress 
















Positive school progress: parent report 


0.4 


0.4 


0.0 


0.624 


5.9 


0.07 




Receives gifted services: teacher report 


1.1 


1.1 


0.0 


0.637 


2.6 


0.06 




Negative school progress: parent report 
Receives remedial services: 


0.2 


0.2 


0.0 


0.213 


-19.5 


-0.16 




teacher report 


1.5 


1.5 


0.0 


0.749 


2.6 


0.06 




Time absent: teacher report 


2.1 


1.8 


0.3 


0.150 


15.2 


0.25 




Time tardy: teacher report 


1.7 


1.5 


0.2 


0.264 


14.0 


0.19 





SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 6.1 presents the range of outcomes reported in this table. 

For the group with no potential barriers, test results were available for 2 1 6 children; parent reports were available for 
222 children; and child reports were available for 222 childrea For the group with one potential barrier, test results were 
available for 341 children; parent reports were available for 345 children; and child reports were available for 345 children. 
For the group with two potential barriers or more, test results were available for 259 children; parent reports were available 
for 263 children; and child reports were available for 263 children. Actual sample sizes for individual measures may vary 
as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p- value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in in^jacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and f = 

1 0 percent. 
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Appendix Table E.6.3 

Impacts on Children’s Achievement and School Progress, by Ethnicity 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size^ 


P-Value for 
Differences 
Across 
Ethnicities^ 


African-American 

Woodcock-Johnson test of achievement 


Total standard score 


95.1 


93.9 


1.2 


0.372 


1.3 


0.09 


0.775 


Broad reading score 


97.5 


94.9 


2.6 


0.134 


2.8 


0.16 


0.935 


Letter-word score 


96.3 


93.3 


3.0 


0.164 


3.2 


0.15 


0.997 


Conprehension 


98.7 


96.8 


1.8 


0.252 


1.9 


0.11 


0.847 


Broad math score 


92.7 


92.3 


0.4 


0.758 


0.5 


0.03 


0.296 


Calculation 


88.3 


87.6 


0.7 


0.641 


0.8 


0.04 


0.487 


Applied problems 


97.4 


96.9 


0.5 


0.765 


0.5 


0.03 


0.252 


Parent ratings of achievement 


Overall achievement 


3.7 


3.7 


0.1 


0.388 


2.4 


0.08 


0.446 


Reading 


3.7 


3.4 


0.3 *** 


0.005 


9.6 


0.28 


0.497 


Math 


3.7 


3.6 


0.1 


0.318 


2.7 


0.09 


0.106 


Teacher ratings of achievement 


Social Skills Rating System academic subscale 


3.2 


3.1 


0.2 


0.155 


6.1 


0.18 


0.157 


Mock report card - total 


2.9 


2.8 


0.1 


0.527 


3.0 


0.08 


0.720 


Mock report card - reading 


2.8 


2.8 


0.1 


0.609 


2.6 


0.07 


0.889 


Mock report card - math 


2.7 


2.7 


-0.1 


0.696 


-2.0 


-0.05 


0.617 


Classroom Behavior Scale 


3.7 


3.6 


0.1 


0.265 


3.7 


0.13 


0.237 


Teacher expectations for child 


2.9 


2.9 


0.0 


0.901 


-0.6 


-0.02 


0.890 


School progress 


Positive school progress: parent report 


0.5 


0.4 


0.0 


0.147 


10.7 


0.14 


0.842 


Receives gifted services: teacher report 


1.2 


1.1 


0.0 


0.690 


2.2 


0.06 


0.329 


Negative school progress: parent report 


0.2 


0.2 


0.0 


0.355 


-11.4 


-0.09 


0.659 


Receives remedial services: teacher report 


1.5 


1.5 


0.0 


0.902 


0.7 


0.02 


0.435 


Time absent: teacher report 


1.8 


1.8 


0.0 


0.801 


-1.8 


-0.03 


0.093 


Time tardy: teacher report 


1.6 


1.7 


-0.2 


0.285 


-8.9 


-0.14 


0.081 


Hispanic 

Woodcock-Johnson test of achievement 


Total standard score 


98.3 


96.6 


1.8 


0.414 


1.8 


0.12 




Broad reading score 


100.5 


98.4 


2.1 


0.320 


2.2 


0.13 




Letter-word score 


101.8 


98.8 


3.1 


0.247 


3.1 


0.16 




Comprehension 


99.8 


98.0 


1.8 


0.475 


1.9 


0.11 
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Appendix Table E.6.3 (continued) 



Outcome 


Program Control 
Group Group Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size" 


P- Value for 
Differences 
Across 
Ethnicities^ 


Broad math score 


95.0 


93.5 


1.5 


0.565 


1.6 


0.09 




Calculation 


93.2 


89.6 


3.7 


0.161 


4.1 


0.19 




Applied problems 


97.0 


98.6 


-1.6 


0.615 


-1.6 


-0.09 




Parent ratings of achievement 


Overall achievement 


3.9 


3.8 


0.1 


0.387 


3.0 


0.11 




Reading 


3.8 


3.7 


0.1 


0.453 


2.9 


0.09 




^^Math 


4.0 


3.6 


0.3 ** 


0.025 


8.8 


0.30 




Teacher ratings of achievement 


Social Skills Rating System academic subscale 


3.4 


3.3 


0.1 


0.564 


3.1 


0.10 




Mock report card - total 


3.1 


3.1 


0.0 


0.846 


1.1 


0.03 




Mock report card - reading 


3.1 


3.0 


0.1 


0.424 


4.8 


0.13 




Mock report card - math 


3.2 


3.0 


0.1 


0.510 


4.5 


0.12 




Classroom Behavior Scale 


3.7 


3.9 


-0.1 


0.535 


-3.1 


-0.12 




Teacher expectations for child 


2.9 


2.8 


0.1 


0.658 


3.2 


0.08 




School progress 


Positive school progress: parent report 


0.4 


0.3 


0.0 


0.642 


5.9 


0.07 




Receives gifted services: teacher report 


1.1 


1.1 


0.0 


0.826 


-1.6 


-0.04 




Negative school progress: parent report 


0.2 


0.2 


0.0 


0.216 


-24.7 


-0.17 




Receives remedial services: teacher report 


1.3 


1.4 


-0.1 


0.375 


-7.6 


-0.16 




Grade retention: teacher report 


1.8 


1.9 


-0.1 


0.241 


-5.5 


-0.26 




Time absent: teacher report 


2.0 


2.0 


0.1 


0.680 


4.9 


0.09 




Time tardy: teacher report 


1.8 


1.6 


0.1 


0.587 


8.1 


0.12 




White 

Woodcock-Johnson test of achievement 


Total standard score 


97.0 


93.0 


4.0 


0.266 


4.3 


0.28 




Broad reading score 


96.3 


95.0 


1.3 


0.657 


1.4 


0.08 




Letter-word score 


94.9 


92.2 


2.7 


0.438 


2.9 


0.14 




Comprehension 


97.4 


97.7 


-0.3 


0.928 


-0.3 


-0.02 




Broad math score 


96.6 


89.8 


6.8 ♦ 


0.072 


7.6 


0.41 




Calculation 


92.1 


87.4 


4.8 


0.223 


5.5 


0.25 




Applied problems 


100.7 


93.6 


7.2 


0.130 


7.7 


0.42 




Parent ratings of achievement 


Overall achievement 


3.5 


3.7 


-0.3 


0.215 


-7.0 


-0.24 




Reading 


3.6 


3.4 


0.3 


0.268 


8.3 


0.24 




Math 


3.4 


3.7 


-0.3 


0.168 


-7.9 


-0.28 
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Appendix Table £.6.3 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size“ 


P- Value for 
Differences 
Across 
Ethnicities^ 


Teacher ratings of achievement 


Social Skills Rating System academic subscale 


2.8 


3.2 


-0.4 ♦ 


0.089 


-11.7 


-0.37 




Mock report card - total 


2.9 


2.6 


0.3 


0.315 


10.9 


0.28 




Mock report card - reading 


2.8 


2.6 


0.2 


0.426 


8.1 


0.19 




Mock report card - math 


2.8 


2.6 


0.2 


0.436 


7.4 


0.17 




Classroom Behavior Scale 


3.4 


3.7 


-0.3 


0.165 


-9.0 


-0.33 




Teacher expectations for child 


2.7 


2.7 


0.0 


0.893 


-1.3 


-0.03 




School progress 


Positive school progress: parent report 


0.3 


0.3 


0.0 


0.848 


3.9 


0.04 




Receives gifted services: teacher report 


1.0 


1.1 


-0.1 


0.227 


-7.3 


-0.17 




Negative school progress: parent report 


0.3 


0.3 


0.0 


0.775 


8.1 


0.07 




Receives remedial services: teacher report 


1.5 


1.3 


0.2 


0.282 


14.8 


0.30 




Time absent: teacher report 


2.3 


1.6 


0.7 


0.009 


41.8 


0.62 




Time tardy: teacher report 


1.5 


1.0 


0.5 * 


0.080 


43.8 


0.41 





SOURCE: MDRC calculations using data from the New Hope five-year survey. 



NOTES: Statistical significance levels are indicated as = 1 percent, ** = 5 percent, and * = 10 percent 

Table 6. 1 presents the ranges of outcomes reported in this table. 

For African-Americans, test results were available for 458 children; parent reports were available for 462 children; 
and teacher reports were available for 284 children. For Hispanics, test results were available for 229 children; parent 
reports were available for 236 children; and teacher reports were available for 1 56 children. For whites, test results were 
available for 108 children; parent reports were available for 109 children; and teacher reports were available for 76 
children. Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

^A statistical test was conducted to measure whether impacts differed significantly across the subgroiq) dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and t = 
10 percent. 
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The New Hope Project 
Appendix Table £.6.4 

Impacts on Children's Beliefs and Motivation, by Age 



P-Value for 
Difference 





Program Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size® 


Age Groups^ 


Ases 6-8 
















Competency beliefs 
















Child’s self-perceived ability 
















English 


5.7 


5.8 


-0.1 


0.378 


-2.1 


-0.12 


0.387 


Math 


6.0 


6.1 


-0.1 


0.481 


-1.4 


-0.08 


0.912 


Aspirations and expectations 
















Occupational aspiration 


61.4 


62.1 


-0.7 


0.788 


-1.2 


-0.04 


0.883 


Occupational expectation 


61.4 


60.3 


1.1 


0.684 


1.8 


0.05 


0.641 


Sample size 


140 


136 












A 2 es 9-12 
















Competency beliefs 
















Child’s self-perceived ability 
















English 


5.9 


5.9 


0.0 


0.925 


-0.2 


-0.01 




Math 


5.9 


5.9 


0.0 


0.918 


-0.2 


-0.01 




Child efficacy (Hope Scale) 


4.8 


4.8 


0.0 


0.942 


0.2 


0.01 




School engagement 


4.2 


4.2 


0.1 


0.502 


1.7 


0.08 




Aspirations and expectations 
















Educational expectations 
















Complete high school 


4.5 


4.5 


0.0 


0.715 


0.9 


0.05 




Attend college 


4.5 


4.5 


0.0 


0.927 


-0.2 


-0.01 




Complete college 


4.4 


4.2 


0.1 


0.303 


3.5 


0.13 




Occupational aspiration 


68.0 


67.8 


0.2 


0.948 


0.2 


0.01 




Occupational expectation 


69.1 


65.6 


3.5 


0.188 


5.3 


0.17 




Values for the future 
















Work attitude 


4.3 


4.3 


0.0 


0.817 


-0.5 


-0.03 




Future beliefs - individual 


3.9 


3.8 


0.1 


0.379 


1.6 


0.11 




Future beliefs - conununity 


4.6 


4.7 


-0.1 


0.225 


-1.6 


-0.16 




Sample size 


137 


163 












Aces 13-16 
















Competency beliefs 
















Child’s self-perceived ability 
















English 


5.8 


5.7 


0.1 


0.303 


2.6 


0.14 




Math 


5.3 


5.4 


-0.1 


0.765 


-1.0 


-0.05 




Child efficacy (Hope Scale) 


4.6 


4.4 


0.2 * 


0.089 


5.0 


0.24 




School engagement 


3.8 


3.6 


0.3 * 


0.056 


7.4 


0.28 
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Appendix Table £.6.4 (continued) 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for % 

Difference Impact 


Effect 

Size" 


P-Value for 
Difference 
Across 
Age Groups^ 


Aspirations and expectations 
Educational expectations 
Complete high school 


4.8 


4.7 


0.1 


0.567 


1.1 


0.07 




Attend college 


4.3 


4.0 


0.3 ♦ 


0.069 


6.9 


0.28 




Complete college 


4.2 


3.8 


0.4 ♦♦ 


0.023 


9.7 


0.34 




Occupational aspiration 


69.7 


68.5 


1.2 


0.668 


1.8 


0.06 




Occupational expectation 


64.7 


65.0 


-0.3 


0.931 


-0.4 


-0.01 




Values for the future 


Work attitude 


4.2 


4.2 


0.1 


0.496 


1.4 


0.09 




Future beliefs - individual 


4.1 


4.1 


0.0 


0.716 


0.6 


0.04 




Future beliefs - community 


4.6 


4.4 


0.2 ** 


0.013 


4.1 


0.38 




Sample size 


129 


103 













SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 6.3 presents the ranges of outcomes presented in this table. 

Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup 
dimensions featured in this table. This p-value represents the probability that apparent variation in impacts across 
each of these dimensions is simply the result of random chance. If this probability is less than 10 percent, the 
variation in impacts is considered statistically significant. Statistical significance levels are indicated as ttt = 1 
percent, tt = 5 percent, and t = 1 0 percent. 
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The New Hope Project 
Appendix Table £.6.5 



Impacts on Children's Beliefs and Motivation, by Number of Parent’s 
Potential Barriers to Employment 

















P- Value for 
Difference 




Program 


Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference Impact 


Size° 


Barrier Groups^ 


No potential barriers 
















Competency beliefs 
















Child's self-perceived ability 
















English 


5.7 


5.9 


-0.2 


0.354 


-2.6 


-0.15 


0.311 


Math 


5.5 


6.0 


-0.4 *** 


0.003 


-7.5 


-0.43 


0.006 ttt 


Child efficacy (Hope Scale) 


4.8 


4.8 


0.0 


0.973 


-0.1 


-0.01 


0.507 


School engagement 


4.0 


3.8 


0.1 


0.471 


3.3 


0.13 


0.692 


Aspirations and expectations 
















Educational expectations 
















Complete high school 


4.7 


4.6 


0.1 


0.591 


1.9 


0.11 


0.898 


Attend college 


4.3 


4.4 


-0.1 


0.644 


-2.4 


-0.11 


0.423 


Complete college 


4.3 


4.1 


0.2 


0.503 


4.5 


0.17 


0.590 


Occupational aspiration 


66.8 


69.3 


-2.5 


0.406 


-3.6 


-0.13 


0.448 


Occupational expectation 


64.3 


69.0 


-4.6 


0.126 


-6.7 


-0.23 


0.139 


Values for the future 
















Work attitude 


4.2 


4.3 


-0.1 


0.381 


-2.7 


-0.18 


0.399 


Future beliefs - individual 


4.0 


4.0 


0.0 


0.982 


-0.1 


0.00 


0.902 


Future beliefs - community 


4.6 


4.5 


0.1 ♦ 


0.092 


2.9 


0.27 


0.515 


Sample size 


124 


95 












One potential barrier 
















Competency beliefs 
















Child's self-perceived ability 
















English 


5.9 


5.8 


0.1 


0.364 


1.8 


0.10 




Math 


5.9 


5.9 


0.0 


0.784 


-0.5 


-0.03 




Child efficacy (Hope Scale) 


4.7 


4.8 


0.0 


0.900 


-0.4 


-0.02 




School engagement 


4.0 


4.0 


0.0 


0.771 


-0.9 


-0.04 




Aspirations and expectations 
















Educational expectations 
















Complete high school 


4.7 


4.6 


0.0 


0.633 


1.1 


0.06 




Attend college 


4.4 


4.3 


0.1 


0.271 


3.4 


0.15 




Complete college 


4.3 


4.1 


0.2 


0.300 


4.1 


0.15 




Occupational aspiration 


65.8 


63.8 


2.0 


0.423 


3.2 


0.10 




Occupational expectation 


65.6 


62.0 


3.5 


0.181 


5.7 


0.17 
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Appendix Table E.6.5 (continued) 



P-Value for 
Difference 

Program Control P-Value for % Effect Across 



Outcome 


Group 


Group 


Difference 


Difference Impact 


Size® Barrier Groups*’ 


Values for the future 




/ 










Work attitude 


4.3 


4.3 


0.0 


0.883 


-0.3 


-0.02 


Future beliefs - individual 


4.0 


4.0 


0.0 


0.795 


0.5 


0.03 


Future beliefs - community 


4.6 


4.6 


0.0 


0.701 


0.6 


0.06 


Sample size 


172 


168 










Two Dotential barriers or more 














Competency beliefs 














Child's self-perceived ability 














English 


5.9 


5.7 


0.1 


0.352 


2.4 


0.13 


Math 


5.8 


5.6 


0.2 


0.169 


3.7 


0.20 


Child efficacy (Hope Scale) 


4.7 


4.4 


0.2 


0.179 


5.0 


0.24 


School engagement 


4.0 


3.9 


0.1 


0.588 


2.5 


0.10 


Aspirations and expectations 














Educational expectations 














Complete high school 


4.6 


4.5 


0.1 


0.399 


3.0 


0.17 


Attend college 


4.4 


4.2 


0.2 


0.249 


5.1 


0.22 


Complete college 


4.4 


4.0 


0.4 ♦♦ 


0.034 


10.2 


0.37 


Occupational aspiration 


67.2 


66.0 


1.2 


0.644 


1.8 


0.06 


Occupational expectation 


64.2 


63.2 


1.1 


0.731 


1.7 


0.05 


Values for the future 














Work attitude 


4.3 


4.2 


0.1 


0.410 


2.6 


0.17 


Future beliefs - individual 


3.9 


3.8 


0.1 


0.512 


1.6 


0.11 


Future beliefs - community 


4.5 


4.5 


0.0 


0.861 


0.5 


0.04 


Sample size 


117 


142 











SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Table 6.3 presents the range of outcomes reported in this table. 

Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard deviation 
of the outcomes for both groups combined. This standard deviation is always obtained from the entire research sample, even 
if the table shows impacts for subgroups. 

**A statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt == 5 percent, and t = 10 
percent. 
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The New Hope Project 
Appendix Table E.6.6 

Five-Year Impacts on Children's Beliefs and Motivation, by Ethnicity 

















P- Value for 
Differences 




Program 


Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size“ 


Ethnicities^ 


African-American 
















Competency beliefs 
















Child's self-perceived ability 
















English 


5.9 


5.9 


0.0 


0.919 


0.2 


0.01 


0.921 


Math 


5.9 


5.9 


0.0 


0.890 


0.2 


0.01 


0.825 


Child efficacy (Hope Scale) 


4.8 


4.6 


0.2 


0.103 


3.9 


0.20 


0.212 


School engagement 


3.9 


3.8 


0.1 


0.401 


2.6 


0.11 


0.605 


Aspirations and expectations 
















Educational expectations 
















Complete high school 


4.7 


4.6 


0.1 


0.501 


1.2 


0.07 


0.913 


Attend college 


4.6 


4.4 


0.2 ♦♦ 


0.045 


4.8 


0.21 


0.934 


Complete college 


4.4 


4.1 


0.3 ♦♦ 


0.034 


6.9 


0.26 


0.803 


Occupational aspiration 


67.8 


66.8 


1.1 


0.616 


1.6 


0.05 


0.417 


Occupational expectation 


66.9 


64.0 


2.9 


0.182 


4.5 


0.14 


0.177 


Values for the future 
















Work attitude 


4.3 


4.2 


0.0 


0.616 


1.0 


0.07 


0.092 t 


Future beliefs - individual 


4.0 


3.9 


0.1 


0.392 


1.4 


0.10 


0.935 


Future beliefs - community 


4.6 


4.5 


0.1 


0.446 


1.1 


0.11 


0.922 


Sample size 


240 


216 












Hispanic 
















Competency beliefs 
















Child's self-perceived ability 
















English 


5.7 


5.8 


-0.1 


0.726 


-0.9 


-0.05 




Math 


5.6 


5.7 


-0.1 


0.675 


-1.2 


-0.06 




Child efficacy (Hope Scale) 


4.7 


4.8 


-0.1 


0.492 


-2.1 


-0.11 




School engagement 


4.1 


4.1 


0.0 


0.750 


-1.0 


-0.04 




Aspirations and expectations 
















Educational expectations 
















Complete high school 


4.7 


4.6 


0.1 


0.489 


1.9 


0.11 




Attend college 


4.3 


4.2 


0.1 


0.529 


3.1 


0.13 




Complete college 


4.3 


4.1 


0.2 


0.289 


5.9 


0.22 




Occupational aspiration 


65.8 


63.8 


2.0 


0.468 


3.1 


0.10 




Occupational expectation 


63.6 


61.7 


1.9 


0.548 


3.0 


0.09 
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P-Value for 
Differences 




Program 


Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size^ 


Ethnicities^ 


Values for the future 
















Work attitude 


4.3 


4.3 


0.0 


0.864 


-0.5 


-0.03 




Future beliefs - individual 


4.0 


3.9 


0.0 


0.746 


0.7 


0.05 




Future beliefs - community 


4.6 


4.6 


0.0 


0.616 


0.7 


0.07 




Sample size 


113 


119 












White 
















Competency beliefs 
















Child's self-perceived ability 
















English 


5.6 


5.7 


-0.1 


0.806 


-1.3 


-0.07 




Math 


5.7 


5.8 


-0.1 


0.575 


-2.1 


-0.11 




Child efficacy (Hope Scale) 


4.4 


4.7 


-0.3 


0.387 


-6.5 


-0.34 




School engagement 


4.0 


4.2 


-0.2 


0.550 


-3.8 


-0.17 




Aspirations and expectations 
















Educational expectations 
















Complete high school 


4.5 


4.3 


0.2 


0.551 


5.4 


0.29 




Attend college 


4.2 


4.0 


0.2 


0.592 


5.5 


0.23 




Complete college 


4.2 


3.6 


0.6 


0.148 


16.2 


0.53 




Occupational aspiration 


63.9 


68.8 


-5.0 


0.219 


-7.2 


-0.26 




Occupational expectation 


62.9 


69.7 


-6.8 * 


0.095 


-9.8 


-0.34 




Values for the future 
















Work attitude 


4.0 


4.6 


-0.6 **♦ 


0.009 


-12.4 


-0.87 




Future beliefs - individual 


3.9 


3.9 


0.0 


0.940 


-0.5 


-0.03 




Future beliefs - community 


4.5 


4.6 


0.0 


0.947 


-0.2 


-0.02 




Sample size 


45 


62 













SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 6.3 presents the ranges of outcomes reported in this table. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as fft = 1 percent, ft “ 5 percent, and f 
= 1 0 percent. 



- 263 - 



299 



The New Hope Project 
Appendix Table E.6.7 

Impacts on Children's Social and Risky Behavior, by Age 















P-Value for 














Difference 




Program 


Control 


P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference Difference 


Impact 


Size^ 


Age Groups” 



Ages 6-8 



Positive Behavior Scale 



Parent report 


3.9 


3.9 


0.0 


0.709 


0.6 


0.05 


0.421 


Teacher report 


3.7 


3.7 


-0.1 


0.601 


-1.5 


-0.08 


0.556 


Problem Behavior Scale 


Total: parent report 


2.3 


2.3 


0.0 


0.837 


-0.6 


-0.02 


0.504 


Total: teacher report 


2.3 


2.3 


0.0 


0.967 


0.2 


0.01 


0.800 


Externalizing: parent report 


2.2 


2.3 


-0.1 


0.282 


-3.9 


-0.13 


0.907 


Externalizing: teacher report 


2.1 


2.1 


0.0 


0.930 


0.6 


0.01 


0.753 


Internalizing: parent report 


2.4 


2.3 


0.1 


0.215 


4.2 


0.15 


0.032 tt 


Internalizing: teacher report 


2.1 


2.1 


0.0 


0.802 


1.2 


0.04 


0.972 


Disciplinary action: teacher report 


2.9 


2.5 


0.4 ♦ 


0.071 


15.4 


0.28 


0.132 


Social relationships 


Peer relationships: child report 


4.2 


4.3 


-0.1 


0.487 


-1.4 


-0.09 


0.318 


Hostile intent total: child report 


2.9 


3.0 


-0.1 


0.698 


-3.3 


-0.05 


0.896 


Hostile intent - physical 


1.1 


1.3 


-0.2 


0.131 


-17.4 


-0.17 


0.807 


Hostile intent - social 


1.8 


1.7 


0.1 


0.529 


6.3 


0.08 


0.511 


Risky behavior 


Trouble index: parent report 


0.0 


0.0 


0.0 


0.269 


-32.1 


-0.07 


0.559 


Ages 9-12 

Positive Behavior Scale 


Parent report 


3.9 


3.8 


0.1 


0.370 


1.6 


0.11 




Teacher report 


3.6 


3.6 


0.0 


0.947 


0.2 


0.01 




Problem Behavior Scale 


Total: parent report 


2.3 


2.4 


-0.1 ♦ 


0.088 


-4.9 


-0.21 




Total: teacher report 


2.3 


2.2 


0.1 


0.552 


2.6 


0.10 




Externalizing: parent report 


2.3 


2.4 


-0.1 


0.250 


-4.2 


-0.14 




Externalizing: teacher report 


2.1 


2.1 


0.1 


0.587 


3.8 


0.09 




Internalizing: parent report 


2.3 


2.4 


-0.2 ♦♦ 


0.022 


-7.0 


-0.27 




Internalizing: teacher report 


2.3 


2.3 


0.1 


0.620 


2.7 


0.09 




Disciplinary action: teacher report 


2.3 


2.5 


-0.2 


0.320 


-8.3 


-0.15 




Social relationships 


Peer relationships: child report 


4.1 


4.0 


0.1 


0.188 


2.8 


0.16 




Hostile intent total: child report 


3.2 


3.5 


-0.3 


0.279 


-7.5 


-0.13 




Hostile intent - physical 


1.3 


1.4 


-0.1 


0.432 


-9.4 


-0.10 




Hostile intent - social 


2.0 


2.1 


-0.1 


0.353 


-6.9 


-0.11 




Peer conventional behaviors 


3.4 


3.5 


-0.1 


0.274 


-2.7 


-0.12 
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Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size“ 


P-Value for 
Difference 
Across 
Age Groups^ 


Risky behavior 


Trouble index: parent report 


0.1 


0.1 


0.0 


0.572 


12.0 


0.05 




Delinquent behavior: child report 


1.1 


1.1 


0.0 


0.130 


4.1 


0.17 




Peer delinquency 


1.4 


1.3 


0.1 


0.343 


4.4 


0.11 




Aees 13-16 

Positive Behavior Scale 


Parent report 


3.9 


3.7 


0.1 ♦♦ 


0.043 


3.9 


0.28 




Teacher report 


3.5 


3.4 


0.1 


0.356 


3.8 


0.18 




Problem Behavior Scale 


Total: parent report 


2.4 


2.5 


0.0 


0.684 


-1.4 


-0.06 




Total: teacher report 


2.3 


2.3 


0.0 


0.707 


-1.8 


-0.07 




Externalizing: parent report 


2.4 


2.4 


0.0 


0.659 


-1.8 


-0.06 




Externalizing: teacher report 


2.0 


2.1 


-0.1 


0.586 


-3.8 


-0.09 




Internalizing: parent report 


2.4 


2.5 


-0.1 


0.456 


-3.0 


-0.12 




Internalizing: teacher report 


2.4 


2.4 


0.0 


0.787 


1.3 


0.05 




Disciplinary action: teacher report 


2.1 


2.1 


0.0 


0.990 


0.1 


0.00 




Social relationships 


Peer relationships: child report 


4.2 


4.2 


0.0 


0.821 


-0.5 


-0.03 




Hostile intent total: child report 


3.0 


3.2 


-0.2 


0.544 


-5.6 


-0.09 




Hostile intent - physical 


1.1 


1.1 


-0.1 


0.692 


-6.2 


-0.05 




Hostile intent - social 


1.9 


2.0 


-0.1 


0.599 


-5.2 


-0.08 




Peer conventional behaviors 


3.2 


3.1 


0.1 


0.358 


3.3 


0.13 




Risky behavior 


Trouble index: parent report 


0.2 


0.2 


0.0 


0.956 


1.1 


0.01 




Delinquent behavior: child report 


1.2 


1.2 


0.0 


0.470 


2.5 


0.11 




Had sex: child report 


1.4 


1.4 


0.0 


0.974 


-0.3 


0.00 




Got pregnant: child report 


1.1 


1.0 


0.1 


0.315 


4.9 


0.18 




Peer delinquency, child report 


1.5 


1.5 


-0.1 


0.560 


-3.3 


-0.10 
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SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent 
Table 6,5 presents the ranges of outcomes reported in the table. 

For ages 6 to 8, teacher reports were available for 191 children; parent reports were available for 279 children; and 
child reports were available for 279 children. For ages 9 to 12, teacher reports were available for 193 children; parent 
reports were available for 303 children; and child reports were available for 303 children. For ages 13 to 16, teacher 
reports were available for 142 children; parent reports were available for 234 children; and child reports were available for 
234 children. Actual sample sizes for individual measures may vary as a result of missing data. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as fft = 1 percent, ft = 5 percent, and f = 

1 0 percent. 
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Impacts on Children*s Social and Risky Behavior, by Number of Parentis 
Potential Barriers to Employment 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% Effect 
Impact Size^ 


P-Value for 
Difference 
Across 

Barrier Groups^ 


No potential barriers 
Positive Behavior Scale 


Parent report 


3.9 


3.8 


0.0 


0.653 


0.9 0.07 


0.312 


Teacher report 


3.6 


3.6 


0.0 


0.805 


0.9 0.05 


0.222 


Problem Behavior Scale 


Total: parent report 


2.3 


2.3 


0.0 


0.615 


-1.6 -0.07 


0.794 


Total: teacher report 


2.2 


2.3 


-0.1 


0.590 


-2.9 -0.11 


0.620 


Externalizing: parent report 


2.2 


2.3 


-0.1 


0.487 


-2.9 -0.09 


0.897 


Externalizing: teacher report 


2.1 


2.1 


0.0 


0.969 


-0.3 -0.01 


0.926 


Internalizing: parent report 


2.4 


2.4 


0.0 


0.953 


-0.2 -0.01 


0.598 


Internalizing: teacher report 


2.2 


2.3 


-0.1 


0.298 


-5.5 -0.19 


0.456 


Disciplinary action: teacher report 


2.4 


2.5 


-0.1 


0.780 


-2.9 -0.05 


0.702 


Social relationships 


Peer relationships: child report 


4.2 


4.1 


0.1 


0.619 


1.2 0.07 


0.955 


Hostile intent total: child report 


2.8 


3.0 


-0.2 


0.506 


-6.7 -0.10 


0.158 


Hostile intent - physical 


1.0 


1.1 


-0.1 


0.642 


-8.2 -0.07 


0.371 


Hostile intent - social 


1.8 


1.9 


-0.1 


0.554 


-5.7 -0.08 


0.300 


Peer conventional behaviors 


3.4 


3.3 


0.1 


0.410 


3.3 0.14 


0.058 t 


Risky behavior 


Trouble index: parent report 


0.1 


0.1 


0.0 


0.757 


-7.1 -0.03 


0.900 


Delinquent behavior: child report 


1.2 


1.1 


0.1 ♦ 


0.084 


7.4 0.31 


0.482 


Had sex: child report 


1.3 


1.2 


0.1 


0.513 


9.1 0.15 


0.647 


Got pregnant: child report 


1.0 


1.0 


0.0 


0.383 


-2.4 -0.08 


0.394 


Peer delinquency: child report 


1.4 


1.3 


0.0 


0.678 


3.4 0.09 


0.929 


One potential barrier 
Positive Behavior Scale 


Parent report 


4.0 


3.8 


0.1 ♦♦ 


0.030 


3.7 0.27 




Teacher report 
Problem Behavior Scale 


3.7 


3.6 


0.1 


0.146 


3.9 0.20 




Total: parent report 


2.3 


2.4 


-0.1 


0.158 


-4.3 -0.18 




Total: teacher report 


2.2 


2.2 


0.0 


0.922 


-0.4 -0.01 




Externalizing: parent report 


2.3 


2.4 


-0.1 


0.211 


-4.6 -0.15 




Externalizing: teacher report 


2.0 


1.9 


0.1 


0.553 


3.8 0.09 




Internalizing: parent report 


2.3 


2.4 


-0.1 


0.170 


-4.5 -0.18 




Internalizing: teacher report 


2.2 


2.2 


0.0 


0.908 


-0.6 -0.02 




Disciplinary action - teacher report 


2.4 


2.3 


0.1 


0.758 


2.4 0.04 
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P-Value for 
Difference 



Outcome 


Program Control 
Group Group 


Difference 


P-Value for 
Difference 


% Effect 
Impact Size^ 


Across 

Barrier Groups® 


Social relationships 


Peer relationships: child report 


4.2 


4.2 


0.0 


0.830 


0.4 0.03 




Hostile intent total: child report 


3.2 


3.2 


0.1 


0.780 


2.1 0.03 




Hostile intent - physical 


1.2 


1.3 


-0.1 


0.580 


-6.7 -0.07 




Hostile intent - social 


2.0 


1.9 


0.1 


0.414 


6.7 0.10 




Peer conventional behaviors 


3.2 


3.4 


-0.2 


0.108 


-5.4 -0.24 




Risky behavior 


Trouble index: parent report 


0.1 


0.1 


0.0 


0.786 


6.6 0.03 




Delinquent behavior: child 


1.2 


1.1 


0.0 


0.468 


2.6 0.11 




Had sex: child report 


1.3 


1.4 


-0.1 


0.551 


-6.6 -0.12 




Got pregnant: child report 


1.1 


1.1 


0.0 


0.854 


-1.0 -0.04 




Peer delinquency: child report 


1.4 


1.4 


0.0 


0.745 


1.8 0.05 




Two potential barriers or more 
Positive Behavior scale 


Parent report 


3.8 


3.8 


0.0 


0.884 


0.3 0.02 




Teacher report 


3.5 


3.6 


-0.1 


0.277 


-3.3 -0.17 




Problem Behavior Scale 


Total: parent report 


2.4 


2.5 


-0.1 


0.383 


-3.5 -0.15 




Total: teacher report 


2.4 


2.3 


0.1 


0.339 


3.8 0.15 




Externalizing: parent report 


2.3 


2.5 


-0.1 


0.277 


-5.1 -0.18 




Externalizing: teacher report 


2.3 


2.2 


0.1 


0.655 


2.9 0.07 




Internalizing: parent report 


2.4 


2.5 


-0.1 


0.430 


-3.5 -0.13 




Internalizing: teacher report 


2.4 


2.3 


0.1 


0.441 


4.1 0.14 




Disciplinary action - teacher report 


2.7 


2.4 


0.2 


0.285 


9.7 0.17 




Social relationships 


Peer relationships: child report 


4.1 


4.1 


0.0 


0.586 


1.2 0.07 




Hostile intent total: child report 


2.9 


3.5 


-0.6 ♦♦ 


0.024 


-17.5 -0.30 




Hostile intent - physical 


1.1 


1.5 


-0.4 ♦♦ 


0.025 


-26.4 -0.30 




Hostile intent - social 


1.8 


2.0 


-0.2 


0.210 


-11.0 -0.17 




Peer conventional behaviors 


3.4 


3.2 


0.2 


0.183 


5.9 0.25 




Risky behavior 


Trouble index: parent report 


0.1 


0.1 


0.0 


0.839 


-4.8 -0.03 




Delinquent behavior: child 


1.2 


1.2 


0.0 


0.931 


0.5 0.02 




Had sex: child report 


1.3 


1.4 


0.0 


0.983 


-0.3 -0.01 




Got pregnant: child report 


1.1 


1.0 


0.1 


0.332 


11.2 0.38 




Peer delinquency: child report 


1.5 


1.5 


0.0 


0.944 


-0.5 -0.01 
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SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Table 6.5 presents the range of outcomes reported in this table. 

For the group with no potential barriers, teacher reports were available for 139 children; parent reports were 
available for 222 children; and child reports were available for 222 children. For the group with one potential barrier, 
teacher reports were available for 224 children; parent reports were available for 345 children; and child reports were 
available for 345 children. For the group with two potential barriers or more, teacher reports were available for 167 
children; parent reports were available for 263 children; and child reports were available for 263 children. Actual 
sample sizes for individual measures may vary as a result of missing data. 

®The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire 
research sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and 
t = 1 0 percent. 
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Five-Year Impacts on Children*s Social and Risky Behavior, by Ethnicity 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size" 


P-Value for 
Differences 
Across 
Ethnicities^ 


African-American 
















Positive Behavior Scale 
















Parent report 


3.9 


3.8 


0.1 * 


0.071 


2.5 


0.18 


0.721 


Teacher report 


3.6 


3.6 


0.0 


0.781 


0.7 


0.04 


0.782 


Problem Behavior Scale 
















Total: parent report 


2.3 


2.4 


-0.1 ** 


0.024 


-5.4 


-0.23 


0.259 


Total: teacher report 


2.3 


2.3 


0.0 


0.805 


0.8 


0.03 


0.667 


Externalizing: parent report 


2.3 


2.4 


-0.2 ** 


0.033 


-6.5 


-0.22 


0.496 


Externalizing: teacher report 


2.2 


2.2 


0.0 


0.985 


-0.1 


0.00 


0.414 


Internalizing: parent report 


2.3 


2.5 


-0.1 * 


0.078 


-4.6 


-0.18 


0.233 


Internalizing: teacher report 


2.3 


2.2 


0.1 


0.533 


2.5 


0.08 


0.627 


Disciplinary action: teacher report 


2.4 


2.5 


-0.1 


0.731 


-2.2 


-0.04 


0.065 t 


Social relationships 
















Peer relationships: child report 


4.2 


4.2 


0.0 


0.485 


1.2 


0.07 


0.021 tt 


Hostile intent total: child report 


3.0 


3.3 


-0.3 


0.117 


-9.4 


-0.15 


0.224 


Hostile intent - physical 


1.1 


1.3 


-0.1 


0.329 


-10.0 


-0.10 


0.472 


Hostile intent - social 


1.8 


2.0 


-0.2 


0.152 


-9.1 


-0.14 


0.134 


Peer conventional behaviors 


3.4 


3.4 


0.0 


0.592 


1.4 


0.06 


0.193 


Risky behavior 
















Trouble index: parent report 


0.1 


0.1 


0.0 


0.873 


-2.4 


-0.01 


0.908 


Delinquent behavior: child report 


1.2 


1.2 


0.0 


0.880 


-0.4 


-0.02 


0.440 


Peer delinquency 


1.4 


1.5 


-0.1 


0.384 


-8.6 


-0.17 


0.039 tt 


Got pregnant: child report 


1.1 


1.1 


0.0 


0.744 


-1.5 


-0.05 




Peer delinquency: child report 


1.4 


1.4 


0.0 


0.548 


-2.6 


-0.07 


0.639 


Hisoanic 
















Positive Behavior Scale 
















Parent report 


4.0 


3.9 


0.0 


0.715 


0.7 


0.06 




Teacher report 


3.7 


3.7 


0.0 


0.811 


-0.8 


-0.04 




Problem Behavior Scale 
















Total: parent report 


2.3 


2.3 


0.0 


0.773 


1.3 


0.05 




Total: teacher report 


2.3 


2.1 


0.1 -- 


— 0.286 


5.4 


0.19 




Externalizing: parent report 


2.2 


2.2 


0.0 


0.821 


-1.1 


-0.03 




Externalizing: teacher report 


2.0 


1.9 


0.1 


0.375 


6.5 


0.14 




Internalizing: parent report 


2.4 


2.4 


0.1 


0.547 


2.9 


0.11 
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Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size" 


P- Value for 
Differences 
Across 
Ethnicities^ 


Internalizing: teacher report 


2.3 


2.1 


0.2 


0.181 


7.4 


0.23 




Disciplinary action: teacher report 


2.4 


2.1 


0.2 


0.346 


10.6 


0.16 




Social relationships 


Peer relationships: child report 


4.0 


4.2 


-0.1 


0.138 


-3.4 


-0.20 




Hostile intent total: child report 


3.1 


3.0 


0.1 


0.788 


2.8 


0.04 




Hostile intent: physical 


1.1 


1.2 


-0.1 


0.660 


-7.2 


-0.07 




Hostile intent: social 


2.0 


1.8 


0.2 


0.326 


11.2 


0.15 




Peer conventional behaviors 


3.2 


3.4 


-0.2 


0.104 


-7.0 


-0.31 




Risky behavior 


Trouble index: parent report 


0.1 


0.1 


0.0 


0.784 


-9.5 


-0.03 




Delinquent behavior: child report 


1.1 


1.1 


0.0 


0.607 


2.1 


0.09 




Had sex: child report 


1.1 


1.2 


-0.1 


0.438 


-7.6 


-0.13 




Got pregnant: child report 


1.0 


1.0 


0.0 


0.657 


1.9 


0.07 




Peer delinquency: child report 


1.4 


1.3 


0.0 


0.586 


3.4 


0.09 




White 

Positive Behavior Scale 


Parent report 


3.8 


3.7 


0.1 


0.319 


3.2 


0.22 




Teacher report 


3.5 


3.6 


-O.I 


0.459 


-3.0 


-0.15 




Problem Behavior Scale 


Total: parent report 


2.4 


2.5 


-0.1 


0.389 


-4.0 


-0.17 




Total: teacher report 


2.4 


2.3 


0.1 


0.294 


6.2 


0.23 




Externalizing: parent report 


2.5 


2.6 


-0.1 


0.726 


-2.3 


-0.08 




Externalizing: teacher report 


2.2 


1.9 


0.3 


0.184 


16.5 


0.36 




Internalizing: parent report 


2.3 


2.4 


-0.1 


0.274 


-5.2 


-0.20 




Internalizing: teacher report 


2.4 


2.4 


0.0 


0.787 


-2.0 


-0.07 




Disciplinary action: teacher report 


2.9 


2.0 


0.9 *** 


0.002 


45.1 


0.66 




Social relationships 


Peer relationships: child report 


4.3 


4.0 


0.3 *** 


0.001 


8.6 


0.49 




Hostile intent total: child report 


2.8 


3.6 


-0.8 ** 


0.042 


-23.2 


-0.41 




Hostile intent - physical 


1.0 


1.5 


-0.5 * 


0.069 


-34.9 


-0.40 




Hostile intent - social 


1.8 


2.3 


-0.4 * 


0.091 


-19.8 


-0.33 




Peer conventional behaviors 


3.1 


3.4 


-0.2 


0.372 


-6.6 


-0.29 
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Appendix Table E.6.9 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size® 


P- Value for 
Differences 
Across 
Ethnicities^ 


Risky behavior 
Trouble index: parent report 


0.1 


0.1 


0.0 


0.606 


17.8 


O.II 




Delinquent behavior: child report 


1.3 


1.1 


0.2 


0.126 


I5.I 


0.64 




Had sex: child report 


1.7 


1.0 


0.7 ** 


0.036 


64.9 


0.89 




Got pregnant: child report 


1.0 


1.0 


0.0 


0.000 


0.0 


0.00 




Peer delinquency: child report 


1.6 


1.5 


0.1 


0.634 


9.3 


0.26 





SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as ■*"*"*■ = 1 percent, '*"*' = 5 percent, and '•' = 10 percent. 

For Afncan-Americans, parent reports were available for 462 children; teacher reports were available for 284 
children; and child reports were available for 462 children. For Hispanics, parent reports were available for 236 children; 
teacher reports were available for 156 children; and child reports were available for 236 children. For whites, parent 
reports were available for 1 09 children; teacher reports were available for 76 children; and child reports were available for 
109 children. Actual sample sizes for individual measures may vary as a result of missing data. 

Table 6.5 presents the ranges of outcomes reported in this table. 

^he effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

^A statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 10 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ftt = 1 percent, ft = 5 percent, and f = 

1 0 percent. 
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Appendix Table E.6.10 
Impacts on Children's Health, by Age 

















P- Value for 
Difference 




Program 


Control 




P-Value for 


% Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference Impact 


Size“ 


Age Groups*’ 


Ages 6-8 
















General medical conditions 
















General health problems 


1.3 


1.2 


0.1 


0.284 


4.8 


0.14 


0.746 


General behavior problems 


1.2 


1.2 


0.0 


0.590 


2.3 


0.07 


0.430 


Other health condition 


1.1 


1.1 


0.0 


0.789 


0.9 


0.03 


0.323 


Overall health 


4.3 


4.3 


0.0 


0.975 


-0.1 


0.00 


0.059 t 


Health in past year 
















Ovemi^t hospital stay 


1.0 


1.1 


0.0 


0.381 


-2.3 


-0.12 


0.121 


Diagnosed illness 


1.2 


1.1 


0.1 


0.236 


4.5 


0.14 


0.538 


Received prescription drug 


1.2 


1.2 


0.0 


0.678 


1.5 


0.05 


0.619 


Received counseling or therapy 


1.1 


1.1 


0.0 


0.696 


1.3 


0.04 


0.876 


Health care facilities 
















Used particular health care facility 


1.0 


1.0 


0.0 


0.627 


-1.2 


-0.05 


0.955 


Received medical advice over phone 


0.9 


0.9 


0.0 


0.239 


5.6 


0.17 


0.622 


Time between medical visits 
















Time since last doctor visit 


5.4 


5.5 


-0.1 


0.471 


-1.3 


-0.08 


0.850 


Time since last dentist visit 


4.7 


4.7 


0.0 


0.801 


-0.9 


-0.03 


1.000 


Sample size 


143 


141 












Ages 9-12 
















General medical conditions 
















General health problems 


1.2 


1.2 


0.0 


0.906 


0.6 


0.02 




General behavior problems 


1.2 


1.2 


0.0 


0.722 


-1.5 


-0.04 




Other health condition 


1.1 


1.2 


0.0 


0.540 


-2.5 


-0.08 




Overall health 


4.3 


4.3 


0.0 


0.921 


-0.3 


-0.01 




Health in past year 
















Ovemi^t hospital stay 


1.0 


1.0 


0.0 


0.106 


2.9 


0.14 




Diagnosed illness 


1.2 


1.2 


0.0 


0.731 


1.4 


0.04 




Received prescription drug 


1.2 


1.2 


0.0 


0.397 


-3.5 


-0.11 




Received counseling or therapy 


1.1 


1.1 


0.0 


0.440 


2.7 


0.09 




Health care facilities 
















Used particular health care facility 


1.0 


1.0 


0.0 


0.574 


-1.8 


-0.08 




Received medical advice over phone 


0.9 


0.9 


0.0 


0.629 


1.9 


0.06 
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Appendix Table E.6.10 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for % Effect 

Difference Impact Size“ 


P-Value for 
Difference 
Across 
Age Groups^ 


Time between medical visits 
Time since last doctor visit 
Time since last dentist visit 


5.3 

4.8 


5.4 

4.8 


-0.1 

0.0 


0.581 

0.793 


-1.1 -0.07 
-0.8 -0.03 




Sample size 


139 


165 










Aees 13-16 

General medical conditions 


General health problems 


1.3 


1.2 


0.0 


0.777 


1.4 0.04 




General behavior problems 


1.3 


1.2 


0.1 


0.189 


6.4 0.19 




Other health condition 


1.2 


1.1 


0.1 


0.249 


5.9 0.20 




Overall health 


4.4 


4.0 


0.4 ♦♦ 


0.013 


9.1 0.39 




Health in past year 


Overnight hospital stay 


1.0 


1.1 


0.0 


0.274 


-3.1 -0.16 




Diagnosed illness 


1.2 


1.2 


0.0 


0.624 


-2.1 -0.07 




Received prescription drug 


1.2 


1.2 


0.0 


0.578 


-2.3 -0.07 




Received counseling or therapy 


1.2 


1.1 


0.0 


0.362 


3.9 0.13 




Health care facilities 


Used particular health care facility 


0.9 


1.0 


0.0 


0.544 


-2.4 -0.11 




Received medical advice over phone 


0.9 


0.9 


0.0 


0.898 


-0.7 -0.02 




Time between medical visits 


Time since last doctor visit 


5.2 


5.4 


-0.1 


0.269 


-2.8 -0.17 




Time since last dentist visit 


4.7 


4.8 


0.0 


0.804 


-0.9 -0.04 




Sample size 


134 


109 











SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, = 5 percent, and * = 10 percent. 

Table 6.7 presents the ranges of outcomes reported in this table. 

Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p- value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 1 0 percent, the variation in iirpacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, ft = 5 percent, and f = 
1 0 percent. 
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The New Hope Project 
Appendix Table £.6.11 



Impacts on Children's Health, by Number of Parent's Potential Barriers to Employment 

















P-Value for 
















Difference 




Program 


Control 




P-Value for 


% 


Effect 


Across 


Outcome 


Group 


Group 


Difference 


Difference 


Impact 


Size® 


Barrier Groups^ 


No potential barriers 
















General medical conditions 
















General health problems 


1.2 


1.2 


0.0 


0.621 


-3.0 


-0.08 


0.205 


General behavior problems 


1.3 


1.1 


0.2 *** 


0.001 


17.9 


0.47 


0.000 ttt 


Other health condition 


1.1 


1.1 


0.0 


0.842 


-1.0 


-0.03 


0.727 


Overall health 


4.4 


4.1 


0.3 * 


0.056 


6.7 


0.30 


0.079 t 


Health in past year 
















Overnight hospital stay 


1.0 


1.1 


-0.1 * 


0.064 


-6.9 


-0.37 


0.064 t 


Diagnosed illness 


1.2 


1.2 


0.0 


0.962 


-0.2 


-0.01 


0.695 


Received prescription drug 


1.1 


1.2 


-0.1 


0.294 


-4.7 


-0.15 


0.818 


Received counseling or therapy 


1.1 


1.2 


-0.1 


0.116 


-6.3 


-0.23 


0.043 tt 


Health care facilities 
















Used particular health care facility 


1.0 


1.0 


0.0 


0.924 


-0.3 


-0.01 


0.216 


Received medical advice over phone 


0.9 


0.9 


0.0 


0.732 


2.1 


0.06 


0.448 


Time between medical visits 
















Time since last doctor visit 


4.9 


5.4 


-0.5 *** 


0.002 


-8.8 


-0.55 


0.001 ttt 


Time since last dentist visit 


4.7 


4.8 


-0.1 


0.663 


-1.9 


-0.08 


0.100 t 


Sample size 


128 


100 












One potential barrier 
















General medical conditions 
















General health problems 


1.3 


1.2 


0.0 


0.721 


1.8 


0.05 




General behavior problems 


1.2 


1.3 


-0.1 


0.146 


-6.4 


-0.20 




Other health condition 


1.1 


1.1 


0.0 


0.641 


-1.6 


-0.05 




Overall health 


4.3 


4.3 


0.0 


0.952 


0.2 


0.01 




Health in past year 
















Overnight hospital stay 


1.1 


1.0 


0.0 


0.327 


2.0 


0.10 




Diagnosed illness 


1.2 


1.1 


0.0 


0.231 


3.8 


0.11 




Received prescription drug 


1.2 


1.2 


0.0 


0.771 


-1.1 


-0.04 




Received counseling or therapy 


1.2 


1.1 


0.1 * 


0.073 


6.5 


0.22 




Health care facilities 
















Used particular health care facility 


1.0 


1.0 


0.0 


0.730 


1.0 


0.05 




Received medical advice over phone 


1.0 


0.9 


0.1 


0.106 


8.1 


0.24 
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Appendix Table £.6.11 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for % 

Difference Impact 


Effect 

Size“ 


P-Value for 
Difference 
Across 

Barrier Groups^ 


Time between medical visits 
















Time since last doctor visit 


5.4 


5.4 


0.0 


0.768 


0.6 


0.04 




Time since last dentist visit 


4.8 


4.9 


-0.1 


0.575 


-1.7 


-0.07 




Sample size 


179 


175 













Two potential barriers or more 



General medical conditions 



General health problems 


1.3 


1.2 


0.1 


0.178 


8.7 


0.25 


General behavior problems 


1.3 


1.3 


0.0 


0.636 


2.7 


0.08 


Other health condition 


1.2 


1.2 


0.0 


0.700 


2.4 


0.08 


Overall health 


4.2 


4.3 


-0.1 


0.496 


-2.5 


-0.11 


Health in past year 


Overnight hospital stay 


1.0 


1.0 


0.0 


0.622 


0.9 


0.05 


Diagnosed illness 


1.2 


1.2 


0.0 


0.937 


-0.3 


-0.01 


Received prescription drug 


1.2 


1.2 


0.0 


0.557 


-2.6 


-0.08 


Received counseling or therapy 


1.1 


1.1 


0.0 


0.353 


3.9 


0.13 


Health care facilities 


Used particular health care facility 


0.9 


1.0 


-0.1 


0.154 


-5.5 


-0.25 


Received medical advice over phone 


0.9 


0.9 


0.0 


0.703 


1.6 


0.05 


Time between medical visits 


Time since last doctor visit 


5.5 


5.4 


0.1 


0.341 


2.2 


0.13 


Time since last dentist visit 


4.8 


4.5 


0.3 


0.134 


6.8 


0.26 


Sample size 


122 


147 











SOURCE: MDRC calculations using data from the New Hope five-year survey. 



» NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 6.7 presents the range of outcomes reported in this table. 

Actual sample sizes for individual measures may vary as a result of missing data. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p- value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 1 0 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ftt = I percent, ft = 5 percent, and t = 
1 0 percent. 
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Appendix Table £.6.12 
Impacts on Children's Health, by Ethnicity 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P-Value for 
Difference 


% 

Impact 


Effect 

Size^ 


P-Value for 
Differences 
Across 
Ethnicities^ 


Africa n-America n 
General medical conditions 


General health problems 


1.2 


1.2 


0.0 


0.873 


0.6 


0.02 


0.359 


General behavior problems 


1.2 


1.2 


0.0 


0.547 


2.4 


0.07 


0.923 


Other health condition 


1.2 


1.2 


0.0 


0.771 


-1.1 


-0.04 


0.057 t 


Overall health 


4.3 


4.2 


0.1 


0.376 


2.2 


0.10 


0.857 


Health in past year 


Overnight hospital stay 


1.0 


1.0 


0.0 


0.733 


-0.6 


-0.03 


0.746 


Diagnosed illness 


1.2 


1.2 


0.0 


0.717 


-1.1 


-0.03 


0.432 


Received prescription drug 


1.1 


1.2 


-0.1 


0.159 


-4.5 


-0.14 


0.365 


Received counseling or therapy 


1.1 


1.1 


0.0 


0.479 


-1.8 


-0.06 


0.097 t 


Health care facilities 


Used particular health care facility 


0.9 


0.9 


0.0 


0.669 


-1.4 


-0.06 


0.865 


Received medical advice over phone 


0.9 


0.9 


0.0 


0.401 


3.4 


0.10 


0.638 


Time between medical visits 


Time since last doctor visit 


5.3 


5.4 


-0.1 


0.327 


-1.9 


-0.11 


0.598 


Time since last dentist visit 


4.6 


4.7 


-0.1 


0.489 


-2.0 


-0.08 


0.251 


Sample size 


250 


223 













Hispanic 



General medical conditions 



General health problems 


1.3 


1.2 


0.1 


0.146 


8.9 


0.25 


General behavior problems 


1.2 


1.2 


0.0 


0.469 


4.1 


0.11 


Other health condition 


1.1 


1.1 


0.0 


0.387 


-3.8 


-0.12 


Overall health 


4.3 


4.3 


0.0 


0.937 


‘ 0.3 


0.01 


Health in past year 


Overnight hospital stay 


1.1 


1.1 


0.0 


0.534 


-2.2 


-0.11 


Diagnosed illness 


1.2 


1.1 


0.0 


0.597 


2.4 


0.08 


Received prescription drug 


1.1 


1.2 


0.0 


0.904 


-0.5 


-0.02 


Received counseling or therapy 


1.1 


1.1 


0.1 * 


0.053 


7.9 


0.26 


Health care facilities 


Used particular health care facility 


1.0 


1.0 


0.0 


0.251 


-3.0 


-0.14 


Received medical advice over phone 


0.9 


0.9 


0.0 


0.869 


0.8 


0.02 
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Appendix Table E.6.12 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


P- Value for 
Difference 


% 

Impact 


Effect 

Size“ 


P-Value for 
Differences 
Across 
Ethnicities^ 


Time between medical visits 


Time since last doctor visit 


5.3 


5.5 


-0.2 


0.200 


-3.2 


-0.20 




Time since last dentist visit 


5.0 


5.0 


-0.1 


0.631 


-1.6 


-0.07 




Sample size 


120 


125 












White 

General medical conditions 


General health problems 


1.4 


1.3 


0.1 


0.574 


5.2 


0.16 




General behavior problems 


1.4 


1.4 


0.0 


0.959 


0.4 


0.02 




Other health condition 


1.3 


1.1 


0.1 


0.109 


12.6 


0.41 




Overall health 


4.1 


4.1 


0.0 


0.849 


1.0 


0.04 




Health in past year 


Overnight hospital stay 


1.0 


1.1 


0.0 


0.338 


-3.8 


-0.19 




Diagnosed illness 


1.3 


1.2 


0.1 


0.177 


9.0 


0.29 




Received prescription drug 


1.4 


1.3 


0.1 


0.411 


6.5 


0.22 




Received counseling or therapy 


1.3 


1.3 


0.0 


0.683 


3.9 


0.15 




Health care facilities 


Used particular health care facility 


1.0 


1.0 


0.0 


0.558 


-2.6 


-0.12 




Received medical advice over phone 


1.0 


0.9 


0.1 


0.389 


10.1 


0.30 




Time between medical visits 
Time since last doctor visit 


5.5 


5.5 


0.0 


0.884 


0.5 


0.03 




Time since last dentist visit 


4.9 


4.4 


0.4 


0.226 


10.2 


0.38 




Sample size 


44 


64 













SOURCE: MDRC calculations using data from the New Hope five-year survey. 

NOTES: Statistical significance levels are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Table 6.7 presents the ranges of outcomes reported in this table. 

“The effect size is the difference between program and control group outcomes as a proportion of the standard 
deviation of the outcomes for both groups combined. This standard deviation is always obtained from the entire research 
sample, even if the table shows impacts for subgroups. 

statistical test was conducted to measure whether impacts differed significantly across the subgroup dimensions 
featured in this table. This p-value represents the probability that apparent variation in impacts across each of these 
dimensions is simply the result of random chance. If this probability is less than 1 0 percent, the variation in impacts is 
considered statistically significant. Statistical significance levels are indicated as ttt = 1 percent, tt = 5 percent, and t = 
1 0 percent. 
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Ben Glispie. 

Employment Retention and Advancement 
Project 

Conceived and funded by the U.S. Department of 
Health and Human Services (HHS), this demon- 
stration project is aimed at testing various ways to 
help low-income people find, keep, and advance in 
jobs. 

New Strategies to Promote Stable Employment and 
Career Progression: An Introduction to the 
Employment Retention and Advancement Project 
(HHS). 2002. Dan Bloom, Jacquelyn Anderson, 
Melissa Wavelet, Karen Gardiner, Michael 
Fishman. 

Time Limits 

Welfare Time Limits: State Policies, Implementation, 
and Effects on Families. 2002. Dan Bloom, Mary 
Farrell, Barbara Fink. 

Leavers, Stayers, and Cyclers: An Analysis of the 
Welfare Caseload. 2002. Cynthia Miller. 

Florida’s Family Transition Program 

An evaluation of Florida’s initial time-limited 
welfare program, which includes services, 
requirements, and financial work incentives intended 
to reduce long-term welfare receipt and help welfare 
recipients find and keep jobs. 

The Family Transition Program: Implementation and 
Three-Year Impacts of Florida *s Initial Time- 
Limited Welfare Program. 1999. Dan Bloom, Mary 
Farrell, James Kemple, Nandita Verma. 

The Family Transition Program: Final Report on 
Florida 's Initial Time-Limited Welfare Program. 
2000. Dan Bloom, James Kemple, Pamela Morris, 
Susan Scrivener, Nandita Verma, Richard Hendra. 



Cross-State Study of Time-Limited Welfare 

An examination of the implementation of some of the 
first state-initiated time-limited welfare programs. 

Welfare Time Limits: An Interim Report Card. 1999. 
Dan Bloom. 

Connecticut’s Jobs First Program 

An evaluation of Connecticut’s statewide time- 
limited welfare program, which includes financial 
work incentives and requirements to participate in 
employment-related services aimed at rapid job 
placement. This study provides some of the earliest 
information on the effects of time limits in major 
urban areas. 

Connecticut Post-Time Limit Tracking Study: Six- 
Month Survey Results. 1999. Jo Anna Hunter- 
Manns, Dan Bloom. 

Jobs First: Implementation and Early Impacts of 
Connecticut’s Welfare Reform Initiative. 2000. Dan 
Bloom, Laura Melton, Charles Michalopoulos, 
Susan Scrivener, Johanna Walter. 

Connecticut ’s Jobs First Program: An Analysis of 
Welfare Leavers. 2000. Laura Melton, Dan Bloom. 
Final Report on Connecticut ’s Welfare Reform 
Initiative. 2002. Dan Bloom, Susan Scrivener, 
Charles Michalopoulos, Pamela Morris, Richard 
Hendra, Diana Adams-Ciardullo, Johanna Walter. 

Vermont’s Welfare Restructuring Project 

An evaluation of Vermont’s statewide welfare reform 
program, which includes a work requirement after a 
certain period of welfare receipt, and financial work 
incentives. 

Forty-Two-Month Impacts of Vermont’s Welfare 
Restructuring Project. 1999. Richard Hendra, 
Charles Michalopoulos. 

WRP: Key Findings from the Forty-Two-Month 
Client Survey. 2000. Dan Bloom, Richard Hendra, 
Charles Michalopoulos. 

WRP: Final Report on Vermont’s Welfare 
Restructuring Project. 2002. Susan Scrivener, 
Richard Hendra, Cindy Redcross, Dan Bloom, 
Charles Michalopoulos, Johanna Walter. 

Financial Incentives 

Encouraging Work, Reducing Poverty: The Impact of 
Work Incentive Programs. 2000. Gordon Berlin. 

Minnesota Family Investment Program 

An evaluation of Minnesota’s pilot welfare reform 
initiative, which aims to encourage work, alleviate 
poverty, and reduce welfare dependence. 

Reforming Welfare and Rewarding Work: Final 
Report on the Minnesota Family Investment 
Program. 2000: 
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Volume I: Effects on Adults. Cynthia Miller, 
Virginia Knox, Lisa Gennetian, Martey Dodoo, 
Jo Anna Hunter, Cindy Redcross. 

Volume 2: Effects on Children. Lisa Gennetian, 
Cynthia Miller. 

Reforming Welfare and Rewarding Work: A 
Summary of the Final Report on the Minnesota 
Family Investment Program. 2000. Virginia Knox, 
Cynthia Miller, Lisa Gennetian. 

Final Report on the Implementation and Impacts of 
the Minnesota Family Investment Program in 
Ramsey County. 2000. Patricia Auspos, Cynthia 
Miller, Jo Anna Hunter. 

New Hope Project 

A test of a community-based, work- foe used 
antipoverty program and welfare alternative operating 
in Milwaukee. 

New Hope for People with Low Incomes: Two-Year 
Results of a Program to Reduce Poverty and 
Reform Welfare. 1999. Johannes Bos, Aletha 
Huston, Robert Granger, Greg Duncan, Thomas 
Brock, Vonnie McLoyd. 

Canada’s Self-Sufficiency Project 

A test of the effectiveness of a temporary earnings 
supplement on the employment and welfare receipt 
of public assistance recipients. Reports on the Self- 
Sufficiency Project are available from: Social 
Research and Demonstration Corporation (SRDC), 
275 Slater St., Suite 900, Ottawa, Ontario KIP 5H9, 
Canada. Tel.: 613-237-431 1; Fax: 613-237-5045. In 
the United States, the reports are also available from 
MDRC. 

Does SSP Plus Increase Employment? The Effect of 
Adding Services to the Self-Sufficiency Project’s 
Financial Incentives (SRDC). 1999. Gail Quets, 
Philip Robins, Elsie Pan, Charles Michalopoulos, 
David Card. 

When Financial Work Incentives Pay for 
Themselves: Early Findings from the Self- 
Sufficiency Project’s Applicant Study (SRDC). 
1999. Charles Michalopoulos, Philip Robins, 

David Card. 

The Self-Sufficiency Project at 36 Months: Effects of 
a Financial Work Incentive on Employment and 
Income (SRDC). 2000. Charles Michalopoulos, 
David Card, Lisa Gennetian, Kristen Harknett, 
Philip K. Robins. 

The Self-Sufficiency Project at 36 Months: Effects on 
Children of a Program That Increased Parental 
Employment and Income (SRDC). 2000. Pamela 
Morris, Charles Michalopoulos. 

When Financial Incentives Pay for Themselves: 
Interim Findings from the Self-Sufficiency 
Project ’s Applicant Study (SRDC). 2001. Charles 
Michalopoulos, Tracey Hoy. 







SSP Plus at 36 Months: Effects of Adding 
Employment Services to Financial Work Incentives 
(SRDC). 2001. Ying Lei, Charles Michalopoulos. 
Making Work Pay: Final Report on the Self- 
Sufficiency Project for Long-Term Welfare 
Recipients (SRDC). 2002. Charles Michalopoulos, 
Doug Tattrie, Cynthia Miller, Philip Robins, 
Pamela Morris, David Gyarmati, Cindy Redcross, 
Kelly Foley, Reuben Ford. 

Mandatory Welfare Employment Programs 

National Evaluation of Welfare-to-Work 
Strategies 

Conceived and sponsored by the U.S. Department of 
Health and Human Services (HHS), with support 
from the U.S. Department of Education (ED), this is 
the largest-scale evaluation ever conducted of 
different strategies for moving people from welfare 
to employment. 

Do Mandatory Welfare-to-Work Programs Affect the 
Well-Being of Children? A Synthesis of Child 
Research Conducted as Part of the National 
Evaluation of Welfare-to- Work Strategies 
(HHS/ED). 2000. Gayle Hamilton. 

Evaluating Alternative Welfare-to- Work Approaches: 
Two-Year Impacts for Eleven Programs 
(HHS/ED). 2000. Stephen Freedman, Daniel 
Friedlander, Gayle Hamilton, JoAnn Rock, Marisa 
Mitchell, Jodi Nudelman, Amanda Schweder, 

Laura Storto. 

Impacts on Young Children and Their Families Two 
Years After Enrollment: Findings from the Child 
Outcomes Study (HHS/ED). 2000. Sharon 
McGroder, Martha Zaslow, Kristin Moore, Suzanne 
LeMenestrel. 

What Works Best for Whom: Impacts of 20 Welfare-to- 
Work Programs by Subgroup (HHS/ED). 2000. 
Charles Michalopoulos, Christine Schwartz. 
Evaluating Two Approaches to Case Management: 
Implementation, Participation Patterns, Costs, and 
Three-Year Impacts of the Columbus Welfare-to- 
Work Program (HHS/ED). 2001. Susan Scrivener, 
Johanna Walter. 

How Effective Are Different Welfare-to- Work 
Approaches? Five-Year Adult and Child Impacts for 
Eleven Programs - Executive Summary (HHS/ED). 
2001. Gayle Hamilton, Stephen Freedman, Lisa 
Gennetian, Charles Michalopoulos, Johanna Walter, 
Diana Adams-Ciardullo, Anna Gassman-Pines, 
Sharon McGroder, Martha Zaslow, Suijeet 
Ahluwalia, Jennifer Brooks. 

Moving People from Welfare to Work: Lessons from 
the National Evaluation of Welfare-to-Work 
Strategies (HHS/ED). 2002. Gayle Hamilton. 
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Los Angeles’s Jobs-First GAIN Program 

An evaluation of Los Angeles’s refocused GAIN 
(welfare-to-work) program, which emphasizes rapid 
employment. This is the first in-depth study of a full- 
scale “work first” program in one of the nation’s 
largest urban areas. 

The Los Angeles Jobs-First GAIN Evaluation: First- 
Year Findings on Participation Patterns and 
Impacts. 1999. Stephen Freedman, Marisa 
Mitchell, David Navarro. 

The Los Angeles Jobs-First GAIN Evaluation: Final 
Report on a Work First Program in a Major Urban 
Center. 2000. Stephen Freedman, Jean Knab, Lisa 
Gennetian, David Navarro. 

Teen Parents on Welfare 

Teenage Parent Programs: A Synthesis of the Long- 
Term Effects of the New Chance Demonstration, 
Ohio 's Learning, Earning, and Parenting (LEAP) 
Program, and the Teenage Parent Demonstration 
(TPD). 1998. Robert Granger, Rachel Cytron. 

Ohio’s LEAP Program 

An evaluation of Ohio’s Learning, Earning, and 
Parenting (LEAP) Program, which uses financial 
incentives to encourage teenage parents on welfare to 
stay in or return to school. 

LEAP: Final Report on Ohio *s Welfare Initiative to 
Improve School Attendance Among Teenage 
Parents. 1997. Johannes Bos, Veronica Fellerath. 

New Chance Demonstration 

A test of a comprehensive program of services that 
seeks to improve the economic status and general 
well-being of a group of highly disadvantaged young 
women and their children. 

New Chance: Final Report on a Comprehensive 
Program for Young Mothers in Poverty and Their 
Children. 1997. Janet Quint, Johannes Bos, Denise 
Polit. 

Parenting Behavior in a Sample of Young Mothers in 
Poverty : Results of the New Chance Observational 
Study. 1998. Martha Zaslow, Carolyn Eldred, 
editors. 

Center for Employment Training 
Replication 

This study is testing whether the successful results 
for youth of a training program developed in San 
Jose can be replicated in 12 other sites around the 
country. 



Evaluation of the Center for Employment Training 
Replication Sites: Interim Report (Berkeley Policy 
Associates). 2000. Stephen Walsh, Deana 
Goldsmith, Yasuyo Abe, Andrea Cann. 

Focusing on Fathers 

Parents’ Fair Share Demonstration 

A demonstration for unemployed noncustodial 
parents (usually fathers) of children on welfare. PFS 
aims to improve the men’s employment and earnings, 
reduce child poverty by increasing child support 
payments, and assist the fathers in playing a broader 
constructive role in their children’s lives. 

Fathers ' Fair Share: Helping Poor Men Manage 
Child Support and Fatherhood (Russell Sage 
Foundation). 1999. Earl Johnson, Ann Levine, 

Fred Doolittle. 

Parenting and Providing: The Impact of Parents ' 
Fair Share on Paternal Involvement. 2000. 

Virginia Knox, Cindy Redcross. 

Working and Earning: The Impact of Parents ' Fair 
Share on Low-Income Fathers * Employment. 2000. 
John M. Martinez, Cynthia Miller. 

The Responsible Fatherhood Curriculum. 2000. 

Eileen Hayes, with Kay Sherwood. 

The Challenge of Helping Low-Income Fathers 
Support Their Children: Final Lessons from 
Parents ' Fair Share. 2001. Cynthia Miller, 

Virginia Knox. 

Career Advancement and Wage 
Progression 

Opening Doors to Earning Credentials 

An exploration of strategies for increasing low- wage 
workers’ access to and completion of community 
college programs. 

Opening Doors: Expanding Educational Oppor- 
tunities for Low-Income Workers. 2001. Susan 
Golonka, Lisa Matus-Grossman. 

Welfare Reform and Community Colleges: A Policy 
and Research Context. 2002. Thomas Brock, Lisa 
Matus-Grossman, Gayle Hamilton. 

Opening Doors: Students * Perspectives on Juggling 
Work, Family, and College. 2002. Lisa Matus- 
Grossman, Susan Gooden. 

Opening Doors: Supporting CalWORKs Students at 
California Community Colleges: An Exploratory 
Focus Group Study. 2002. Laura Nelson, Rogeair 
Purnell. 
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Education Reform 

Career Academies 

The largest and most comprehensive evaluation of a 
school-to-work initiative, this study examines a 
promising approach to high school restructuring and 
the school-to-work transition. 

Career Academies: Building Career Awareness and 
Work- Based Learning Activities Through Employer 
Partnerships. 1999. James Kemple, Susan 
Poglinco, Jason Snipes. 

Career Academies: Impacts on Students* 

Engagement and Performance in High School. 
2000. James Kemple, Jason Snipes. 

Career Academies: Impacts on Students ' Initial 
Transitions to Post-Secondary Education and 
Employment. 2001. James Kemple. 

First Things First 

This demonstration and research project looks at First 
Things First, a whole-school reform that combines a 
variety of best practices aimed at raising achievement 
and graduation rates in both urban and rural settings. 

Scaling Up First Things First: Site Selection and the 
Planning Year. 2002. Janet Quint. 

Closing Achievement Gaps 

Conducted for the Council of the Great City Schools, 
this study identifies districtwide approaches to urban 
school reform that appear to raise overall student 
performance while reducing achievement gaps 
among racial groups. 

Foundations for Success: Case Studies of How 
Urban School Systems Improve Student 
Achievement. 2002. Jason Snipes, Fred Doolittle, 
Corinne Herlihy. 

Project GRAD 

This evaluation examines Project GRAD, an 
education initiative targeted at urban schools and 
combining a number of proven or promising reforms. 

Building the Foundation for Improved Student 
Performance: The Pre-Curricular Phase of Project 
GRAD Newark. 2000. Sandra Ham, Fred Doolittle, 
Glee Ivory Holton. 

Accelerated Schools 

This study examines the implementation and impacts 
on achievement of the Accelerated Schools model, a 
whole-school reform targeted at at-risk students. 

Evaluating the Accelerated Schools Approach: A 
Look at Early Implementation and Impacts on 
Student Achievement in Eight Elementary Schools. 
2001. Howard Bloom, Sandra Ham, Laura Melton, 
Julienne O’Brien. 



Extended-Service Schools Initiative 

Conducted in partnership with Public/Private 
Ventures (P/PV), this evaluation of after-school 
programs operated as part of the Extended-Service 
Schools Initiative examines the programs’ implemen- 
tation, quality, cost, and effects on students. 

Multiple Choices After School: Findings from the 
Extended-Service Schools Initiative (P/PV). 2002. 
Jean Baldwin Grossman, Marilyn Price, Veronica 
Fellerath, Linda Jucovy, Lauren Kotloff, Rebecca 
Raley, Karen Walker. 

School-to-Work Project 

A study of innovative programs that help students 
make the transition from school to work or careers. 

Home-Grown Lessons: Innovative Programs Linking 
School and IFbr/: (Jos sey-B ass Publishers). 1995. 
Edward Pauly, Hilary Kopp, Joshua Haimson. 
Home-Grown Progress: The Evolution of Innovative 
School-to-Work Programs. 1997. Rachel Pedraza, 
Edward Pauly, Hilary Kopp. 

Project Transition 

A demonstration program that tested a combination 
of school-based strategies to facilitate students’ 
transition from middle school to high school. 

Project Transition: Testing an Intervention to Help 
High School Freshmen Succeed. 1999.Janet Quint, 
Cynthia Miller, Jennifer Pastor, Rachel Cytron. 

Equity 2000 

Equity 2000 is a nationwide initiative sponsored by 
the College Board to improve low-income students’ 
access to college. The MDRC paper examines the 
implementation of Equity 2000 in Milwaukee Public 
Schools. 

Getting to the Right Algebra: The Equity 2000 
Initiative in Milwaukee Public Schools. 1999. 

Sandra Ham, Erica Walker. 

Employment and Community 
Initiatives 

Jobs-Plus Initiative 

A multisite effort to greatly increase employment 
among public housing residents. 

Mobilizing Public Housing Communities for Work: 
Origins and Early Accomplishments of the Jobs- 
Plus Demonstration. 1999. James Riccio. 

Building a Convincing Test of a Public Housing 
Employment Program Using Non-Experimental 
Methods: Planning for the Jobs-Plus 
Demonstration. 1999. Howard Bloom. 
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Jobs-Plus Site-by-Site: An Early Look at Program 
Implementation. 2000. Edited by Susan Philipson 
Bloom with Susan Blank. 

Building New Partnerships for Employment: 
Collaboration Among Agencies and Public 
Housing Residents in the Jobs-Plus 
Demonstration. 2001. Linda Kato, James Riccio. 
Making Work Pay for Public Housing Residents: 
Financial-Incentive Designs at Six Jobs-Plus 
Demonstration Sites. 2002. Cynthia Miller, James 
Riccio. 

The Special Challenges of Offering Employment 
Programs in Culturally Diverse Communities: The 
Jobs-Plus Experience in Public Housing 
Developments. 2002. Linda Kato. 

The Employment Experiences of Public Housing 
Residents: Findings from the Jobs-Plus Baseline 
Survey. 2002. John Martinez. 

Children in Public Housing Developments: An 
Examination of the Children at the Beginning of 
the Jobs-Plus Demonstration. 2002. Pamela 
Morris, Stephanie Jones. 

Jobs-Plus Site-by-Site: Key Features of Mature 
Employment Programs in Seven Public Housing 
Communities. 2003. Linda Kato. 

Staying or Leaving: Lessons from Jobs-Plus About the 
Mobility of Public Housing Residents and 
Implications for Place-Based Initiatives. 2003. 
Nandita Verma. 

Neighborhood Jobs Initiative 

An initiative to increase employment in a number of 
low-income communities. 

The Neighborhood Jobs Initiative: An Early Report 
on the Vision and Challenges of Bringing an 
Employment Focus to a Community-Building 
Initiative. 2001. Frieda Molina, Laura Nelson. 
Structures of Opportunity: Developing the 
Neighborhood Jobs Initiative in Fort Worth, Texas. 
2002. Tony Proscio. 

Final Report on the Neighborhood Jobs Initiative: 
Lessons and Implications for Future Community 
Employment Initiatives. 2003. Frieda Molina, Craig 
Howard. 

Connections to Work Project 

A study of local efforts to increase competition in the 
choice of providers of employment services for 
welfare recipients and other low-income populations. 
The project also provides assistance to cutting-edge 
local initiatives aimed at helping such people access 
and secure jobs. 

Designing and Administering a Wage-Paying 
Community Service Employment Program Under 
TANF: Some Considerations and Choices. 1999. 
Kay Sherwood. 



San Francisco Works: Toward an Employer-Led 
Approach to Welfare Reform and Workforce 
Development. 2000. Steven Bliss. 

Canada’s Earnings Supplement Project 

A test of an innovative financial incentive intended to 
expedite the reemployment of displaced workers and 
encourage fiill-year work by seasonal or part-year 
workers, thereby also reducing receipt of 
unemployment insurance. 

Testing a Re-Employment Incentive for Displaced 
Workers: The Earnings Supplement Project. 1999. 
Howard Bloom, Saul Schwartz, Susanna Lui-Gurr, 
Suk-Won Lee. 

MDRC Working Papers on 
Research Methodology 

A series of papers that explore alternative methods of 
examining the implementation and impacts of 
programs and policies. 

Building a Convincing Test of a Public Housing 
Employment Program Using Non- Experimental 
Methods: Planning for the Jobs-Plus 
Demonstration. 1999. Howard Bloom. 

Estimating Program Impacts on Student Achievement 
Using ** Short” Interrupted Time Series. 1999. 
Howard Bloom. 

Using Cluster Random Assignment to Measure 
Program Impacts: Statistical Implications for the 
Evaluation of Education Programs. 1999. Howard 
Bloom, Johannes Bos, Suk-Won Lee. 

The Politics of Random Assignment: Implementing 
Studies and Impacting Policy. 2000. Judith 
Gueron. 

Assessing the Impact of Welfare Reform on Urban 
Communities: The Urban Change Project and 
Methodological Considerations. 2000. Charles 
Michalopoulos, Joannes Bos, Robert Lalonde, 
Nandita Verma. 

Measuring the Impacts of Whole School Reforms: 
Methodological Lessons from an Evaluation of 
Accelerated Schools. 2001. Howard Bloom. 

A Meta-Analysis of Government Sponsored Training 
Programs. 2001. David Greenberg, Charles 
Michalopoulos, Philip Robins. 

Modeling the Performance of Welfare-to- Work 
Programs: The Effects of Program Management 
and Services, Economic Environment, and Client 
Characteristics. 200 1 . Howard Bloom, Carolyn 
Hill, James Riccio. 

A Regression- Based Strategy for Defining Subgroups 
in a Social Experiment. 2001. James Kemple, 

Jason Snipes. 



Explaining Variation in the Effects of JVelfare-to- 
Work Programs. 2001. David Greenberg, Robert 
Meyer, Charles Michalopoulos, Michael Wiseman. 

Extending the Reach of Randomized Social 
Experiments: New Directions in Evaluations of 
American Welfare-to-Work and Employment 
Initiatives. 2001. James Riccio, Howard Bloom. 

Can Nonexperimental Comparison Group Methods 
Match the Findings from a Random Assignment 
Evaluation of Mandatory Welfare-to-Work 
Programs? 2002. Howard Bloom, Charles 
Michalopoulos, Carolyn Hill, Ying Lei. 



Using Instrumental Variables Analysis to Learn 
More from Social Policy Experiments. 2002. Lisa 
Gennetian, Johannes Bos, Pamela Morris. 

Using Place-Based Random Assignment and 
Comparative Interrupted Time-Series Analysis to 
Evaluate the Jobs-Plus Employment Program for 
Public Housing Residents. 2002. Howard Bloom, 
James Riccio 

Intensive Qualitative Research Challenges, Best 
Uses, and Opportunities. 2003. Alissa Gardenhire, 
Laura Nelson 
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About MDRC 



MDRC is a nonprofit, nonpartisan social policy research organization. We are 
dedicated to learning what works to improve the well-being of low-income people. 
Through our research and the active communication of our findings, we seek to 
enhance the effectiveness of social policies and programs. MDRC was founded in 
1974 and is located in New York City and Oakland, California. 

MDRC’s current projects focus on welfare and economic security, education, and 
employment and community initiatives. Complementing our evaluations of a wide 
range of welfare reforms are new studies of supports for the working poor and 
emerging analyses of how programs affect children’s development and their 
families’ well-being. In the field of education, we are testing reforms aimed at 
improving the performance of public schools, especially in urban areas. Finally, our 
community projects are using innovative approaches to increase employment in 
low-income neighborhoods. 

Our projects are a mix of demonstrations — field tests of promising program 
models — and evaluations of government and community initiatives, and we 
employ a wide range of methods to determine a program’s effects, including large- 
scale studies, surveys, case studies, and ethnographies of individuals and families. 
We share the findings and lessons from our work — including best practices for 
program operators — with a broad audience within the policy and practitioner 
community, as well as the general public and the media. 

Over the past quarter century, MDRC has worked in almost every state, all of the 
nation’s largest cities, and Canada. We conduct our projects in partnership with 
state and local governments, the federal government, public school systems, 
community organizations, and numerous private philanthropies. 




16 East 34thi Street 
New York, NY 10016 
(212) 532-3200 

www.mdrc.org 

475 14th Street 
Oakland, CA 94612 
(510) 663-6372 
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